Inspection device and method

ABSTRACT

This inspection device comprises: image-capturing unit obtaining an evaluation workpiece image that is captured with a plurality of predetermined illumination light emission patterns; image area setting unit setting a plurality of image areas associated with a plurality of different labels; optimization calculation unit generating a first index of which the output value increases as a difference between an image area associated with a non-defective label and a defective label increases, and a second index of which the output value increases as a difference or contrast between the image areas associated with the non-defective label increases, and calculates an illumination light emission pattern for inspection such that the first index becomes larger and the second index becomes smaller; and determination unit performing image processing on an inspection workpiece image captured with the illumination light emission pattern for inspection, and determines a pass/fail label of a workpiece to be inspected.

CROSS-REFERENCE TO RELATED ART

The present application is based on Japanese Patent Application No.2019-048163 filed on Mar. 15, 2019, the entire contents of which areincorporated herein.

TECHNICAL FIELD

The present invention relates to an inspection device for a target and amethod.

BACKGROUND ART

Product appearance inspection in manufacturing sites is one of thefields in which replacement of human workers with machines is leastadvanced, and this is an important problem concerning automation thathas to be addressed with a decrease in working population in the future.In recent years, automation techniques for inspection have beendramatically improved due to development of artificial intelligence andmachine learning techniques, representative examples of which includedeep learning.

However, a process that requires the most time and efforts whenconstructing an inspection system in general appearance inspection andmachine vision is designing of an imaging system including optimizationdesign of an illumination pattern, and automation of this field is notvery advanced. In a case in which persons manually perform optimizationdesign, a large amount of effort is needed to design an imaging systemto reliably detect defects such as scratches that have occurred onworkpieces while addressing variations in individual workpieces (to beinspected). In other words, it is essential to alternately repeatoptimization of illumination through manual adjustment and review andadjustment of inspection algorithms while exchanging various targetworkpieces in order to obtain desired detection performance, and to doso, there is a problem that a significantly large number of processesare needed.

In order to address this problem, a method of optimizing an illuminationpattern to detect defects in workpieces on the basis of a group ofimages captured using a plurality of illumination patterns has beenreported in Patent Literature 1, for example.

CITATION LIST Patent Literature [Patent Literature 1]

EP2887055A1

SUMMARY OF INVENTION Technical Problem

According to such a method in the related art, optimization design of anillumination light emission pattern is performed simply from a viewpointof emphasizing defects in workpieces. In other words, there is atendency that an optimal illumination light emission pattern isdetermined simply on the basis of an index indicating a differencebetween non-defective products with no defects (acceptable products,products with no defects) and defective products with defects(non-acceptable products, products with defects, poor products).However, since there are various appearances and various surface statesdepending on workpieces, and defects that may occur in workpieces havevarious characteristics, there is a concern that it may be difficult todistinguish between background images of the workpiece surfaces anddefects and that portions that are not defects may be determined asdefects when only defects are emphasized. In a case in which, so-calledhairline finishing has been performed on an entire workpiece surface,and linear scratch, for example, has occurred in a part of the surface,for example, there is a concern that a hairline (linear mark) on theworkpiece surface may also be emphasized with an illumination patternfor emphasizing scratching. If this situation happens, the scratch isnot determined as a defect, and there is a likelihood that a defectiveproduct will be missed, or a linear mark in the background will beerroneously determined as a defect.

In other words, a background image for a workpiece surface has randomvariations in a normal portion of a workpiece without defects and randomvariations between individual workpieces with no defects (non-defectiveproducts) (these can also be referred to as “variations in anon-defective product” and “variations in individual products”), and anidea itself of curbing the variations in anon-defective product orvariations in individual products which are workpieces has not beenintroduced to optimization of an illumination pattern in the method inthe related art.

Thus, the present disclosure was made in view of such circumstances inone aspect, and an objective thereof is to provide an inspection method,an inspection device, and the like capable of obtaining an optimizedillumination light emission pattern with which influences of variationsin a non-defective product or variations in individual products whichare workpieces can be curbed while emphasizing defects in workpieces.

Solution to Problem

The present invention employs the following configurations in order tosolve the aforementioned problem.

[1] An inspection device according to an example of the presentdisclosure includes: an image-capturing unit which has a sensor thatimages at least one workpiece illuminated in a plurality ofpredetermined illumination light emission patterns and obtains aplurality of evaluation workpiece images that are associated with thepredetermined illumination light emission patterns and the workpiece; animage area setting unit which sets, for the evaluation workpiece images,a plurality of image areas associated with a plurality of differentlabels that indicate non-defective products or defective products; anoptimization calculation unit which generates, from the image areas, afirst index, an output value of which increases as differences betweenimage areas associated with labels indicating the non-defective productsand image areas associated with labels indicating the defective productsincrease, generates a second index, an output value of which increasesas differences between the image areas associated with the labelsindicating the non-defective products increase or as contrast in theimage areas associated with the labels indicating the non-defectiveproducts increases, and calculates an illumination light emissionpattern for inspection such that (an evaluation value of) the firstindex becomes larger and (an evaluation value of) the second indexbecomes smaller; and a determination unit which performs imageprocessing on an inspection workpiece image obtained by theimage-capturing unit imaging a workpiece to be inspected illuminated inthe illumination light emission pattern for inspection to therebydetermine a pass/fail label of the workpiece to be inspected.

With this configuration, it is possible to extract images in a region ofinterest in the plurality of evaluation workpiece images and to setimage areas (or a group thereof) for the combination of the plurality ofdifferent labels indicating non-defective products or defective-productsand at least one region of interest with a predetermined shape from theevaluation workpiece images obtained by imaging the workpieceilluminated in the plurality of predetermined illumination lightemission patterns. From the obtained image areas, a first indexindicating variations in workpieces belonging to different labels (forexample, variations between non-defective products and defectiveproducts) is set on the basis of varying components between imagesbelonging to different labels, and a second index indicating variationsin workpieces belonging to the same label (for example, variations in anon-defective product or variations in individual products) is set onthe basis of varying components between images belonging to the samelabel. Then, an illumination parameter vector as an optimized solutionthat increases or maximizes the first index and decreases or minimizesthe second index when at least one workpiece is illuminated and imagedin an illumination light emission pattern for evaluation correspondingto an arbitrary combination of a plurality of predetermined illuminationlight emission patterns is obtained. Then, an inspection workpiece imagefor a workpiece to be inspected is acquired using an optimizedillumination light emission pattern obtained on the basis of theillumination parameter vector, and label determination of the workpiece(for example, inspection regarding whether or not there are defects suchas scratches) is performed.

Therefore, it is possible to emphasize defects such as scratches in theworkpiece (to increase the difference between the non-defective productsand defects to allow the defects to be easily recognized), for example,and also to curb influences of random variations in a non-defectiveproduct or variations in individual products caused by a backgroundimage of the workpiece (to reduce variations in a non-defectiveproduct). It is thus possible to reduce missing of defects in workpiecesand determination errors and thereby to improve inspection performanceand inspection efficiency of workpieces. As described above, theinspection device according to the present disclosure is realized forthe first time by newly introducing a point of view of curbingvariations in a non-defective product or variations in individualproducts which are workpieces, which has not been used in the method inthe related art, to optimization of an illumination pattern ininspection of the workpieces.

Note that the “labels” indicate features to be inspected of theworkpieces, the features to be inspected are not particularly limited,and any features to be inspected may be employed as long as the featuresare features of the workpiece 4 that are to be inspected. Examplesthereof include data indicating an appearance feature (for example,scratch, the size, or the like) of the workpiece as well as dataindicating which of a non-defective product (acceptable product) or adefective product (non-acceptable product) the workpiece is, asdescribed above.

[2] More specifically, in the aforementioned configuration, theoptimization calculation unit may be adapted such that the number oflight emission patterns to be evaluated when the illumination lightemission pattern for inspection is calculated is larger than the numberof the predetermined illumination light emission patterns.

[3] Further specifically, in the aforementioned configuration, theoptimization calculation unit may generate an evaluation function on thebasis of the first index and the second index and obtain an optimizedsolution such that when a certain illumination light emission patternfor evaluation is provided, and when the first index becomes a maximumvalue and the second index value becomes a minimum value at the time ofilluminating and imaging the at least one workpiece, an evaluation valueof the evaluation function becomes a maximum.

[4] More specifically, in the aforementioned configuration, theoptimization calculation unit may obtain, through discriminant analysis(DA), an optimized solution that maximizes an evaluation value of anevaluation function (of a ratio type) based on a ratio between the firstindex and the second index or an evaluation value of an evaluationfunction (of a difference type) based on a difference between the firstindex and the second index, for example. Note that as the discriminantanalysis, linear discriminant analysis (LDA) using multivariate analysisis particularly effective, and examples of a specific algorithm thereofinclude the Fisher's linear discriminant analysis (FDLA).

With such a configuration, if imaging using a light source that performsirradiation in a plurality of predetermined illumination light emissionpatterns is regarded as an arithmetic operation, mathematically a simpleinner product arithmetic operation and linear projection themselves areto be performed. According to the linear discriminant analysis, forexample, an arithmetic operation for generating a feature vector is asimple inner product, and it is thus possible to obtain optimal linearprojection with a simple algorithm. Also, if an evaluation functionbased on a ratio between the first index and the second index is used inthe discriminant analysis, it is possible to directly analyzemaximization of the first index and minimization of the second index. Inaddition, if such an evaluation function of a ratio type is used, it ispossible to suitably obtain optimized solutions of both an illuminationlight emission pattern (single shot) based on a single illuminationparameter vector and an illumination light emission pattern (multipleshots) based on a plurality of illumination parameter vectors (that is,an illumination parameter matrix). On the other hand, it is possible toanalyze the maximization of the first index and the minimization of thesecond index using an evaluation function based on a difference betweenthe first index and the second index in the discriminant analysis, andit is possible to obtain an optimal solution of in a single shot, inparticular, at a high speed.

[5] In the aforementioned configuration, the optimization calculationunit may include a restriction condition that all components of theillumination parameter vectors are non-negative values in thediscriminant analysis. For example, an optimal projection direction inthe Fisher's linear discriminant analysis can be obtained as a maximumeigenvector of an eigenvalue in a general eigenvalue problem by aLagrange undetermined multiplier method, and as an illumination pattern,an illumination parameter (intensity) vector may be set in a directionthat follows the eigenvector. However, since it is not possible torealize a negative illumination intensity in actual illuminationalthough components of the eigenvector can be negative values, thispoint may be problematic in optimization of illumination. In thisregard, with this configuration, the restriction condition that all thecomponents of the illumination parameter vector (eigenvector) arenon-negative values is provided in the discriminant analysis, and it isthus possible to obtain an optimized illumination parameter vector, allthe components of which are non-negative values. Note that since it isdifficult to apply a solution as a general eigenvalue problem in thiscase, a solution to which a mathematical programming method, forexample, is applied is effective.

[6] In the aforementioned configuration, in a case in which anillumination parameter vector obtained as the optimized solutionincludes components of positive values and negative values, theoptimization calculation unit may obtain a first optimized illuminationlight emission pattern based on vectors configured with the componentsof the positive values and obtains a second optimized illumination lightemission pattern based on vectors configured with absolute values of thecomponents of the negative values, the image-capturing unit may acquirea first image and a second image by imaging the workpiece to beinspected with illumination in the first optimized illumination lightemission pattern and the second optimized illumination light emissionpattern, respectively, and the determination unit may acquire theinspection workpiece image on the basis of a difference between thefirst image and the second image.

With this configuration, it is possible to obtain a maximum eigenvectorof an eigenvalue in a general eigenvalue problem that can be a negativevalue as an optimized solution that mathematically completely reproducesan illumination intensity of a negative value through pseudoreproduction of the illumination intensity of the negative value, unlike[5] described above. In other words, it is possible to obtain effectsequivalent to those obtained by directly imaging the workpiece to beinspected in an optimized illumination light emission pattern based onthe original optimized illumination parameter vector in which thepositive values and the negative values are present together in a pseudomanner.

[7] In the aforementioned configuration, in a case in which anillumination parameter vector obtained as the optimal solution includescomponents of positive values and negative values, the optimizationcalculation unit may choose, as the illumination parameter vectors, aplurality of eigenvectors with large eigenvalues in main componentanalysis, obtain a first optimized illumination light emission patternbased on vectors configured with the components of the positive values,and obtain a second optimized illumination light emission pattern basedon vectors configured with absolute values of the components of thenegative values for each of the plurality of eigenvectors, theimage-capturing unit may acquire a first image and a second image byimaging the workpiece to be inspected with illumination in the firstoptimized illumination light emission pattern and the second optimizedillumination light emission pattern corresponding to each of theplurality of eigenvectors, and the determination unit may acquire theinspection workpiece image on the basis of a difference between thefirst image and the second image.

Even with this configuration, it is possible to obtain the maximumeigenvector of the eigenvalue in the general eigenvalue problem that canbe a negative value as an optimal solution that mathematicallycompletely reproduces an illumination intensity of the negative valuethrough pseudo reproduction of the illumination intensity of thenegative value, unlike [3] described above. In other words, it ispossible to obtain, in a pseudo manner, effects equivalent to thoseobtained when the workpiece to be inspected is imaged directly in theillumination light emission pattern (multiple shots) of the plurality oforiginal illumination parameter vectors in which positive values andnegative values are present together. Also, since this configurationsubstantially provides multi-shot inspection using a plurality of shots,it is possible to significantly enhance inspection performance ascompared with single-shot inspection. Moreover, since a dark currentoffset value is cancelled by employing the difference between the firstimage and the second image, it is also possible to omit dark currentcorrection if black levels of both the images are the same.

[8] In the aforementioned configuration, in a case in which theillumination light emission pattern for evaluation is for multipleshots, the optimization calculation unit may calculate an optimizedillumination light emission pattern for multiple shots that maximizesthe first index and minimizes the second index when the at least oneworkpiece is illuminated and imaged in an illumination light emissionpattern for evaluation corresponding to one column vector obtained bysuperimposing the plurality of illumination parameter vectorsconfiguring the illumination light emission pattern for evaluation. Withthis configuration, one column vector is an illumination parametermatrix including a plurality of illumination parameter vectors, and itis possible to obtain an illumination light emission pattern for optimalmultiple shots with an arbitrary number of imaged shots.

[9] With the aforementioned configuration, a form of image processingperformed by the determination unit may be linear image processing, andoptimization performed by the optimization calculation unit to maximizethe first index and minimize the second index and optimization of imageprocessing parameters used in image processing performed by thedetermination unit may be performed at the same time. With thisconfiguration, it is possible to handle both single-shot inspection andmulti-shot inspection and to optimize an illumination light emissionpattern used in an inspection stage and linear image processing at thesame time.

[10] In the aforementioned configuration, the optimization calculationunit may perform linear image processing on the evaluation workpieceimages prior to the calculation of the optimized illumination lightemission pattern. It is possible to handle both the single-shotinspection and the multi-shot inspection with this configuration aswell. Also, it is possible to cause images to conform to each other orto be divergent from each other by focusing only on a specific frequencyband, and further, it is possible to suitably cause images to conform toeach other or to be divergent from each other even in a case in whichprocessing of cutting DC components is included to ignore variations inoffset due to a change with time.

[11] With the aforementioned configuration, in the image area settingunit or the optimization calculation unit, both the first index and thesecond index may be total values of varying components of pixel valuesin images in regions of interest (ROI) with the same labels. Asdescribed above, illumination design based on the Fisher's lineardiscrimination, for example, has evaluation criteria that differencesbetween non-defective products and defects are caused to increase toallow the defects to be easily recognized and that variations in anon-defective product are reduced, and this is a method focusing ondifferences among a plurality of images. However, there is a case inwhich it is more convenient to calculate an evaluation value from asingle image and to maximize or minimize the evaluation value (forexample, a case in which it is desired to simply increase or decreasecontrast in a specific region) than to handle differences among images,depending on problem setting. In this regard, since this configurationsubstantially corresponds to evaluation using a single image, theconfiguration is useful for the case in which an evaluation valuecalculated from a single image is maximized or minimized.

[12] In the aforementioned configuration, the optimization calculationunit may obtain a plurality of the optimized illumination light emissionpatterns on the basis of a plurality of illumination parameter vectorsobtained as the optimized solution (suitable solution) in a case inwhich weights are applied to the first index and the second index. Withthis configuration, it is possible to prepare a plurality of optimizedsolutions with clear differences (inflections) through selection orexclusion of conditions and to allow a user to perform comparison andconsideration, for example. Also, it is possible to cause how images areclose to each other (degrees of coincidence) and an evaluation valuefrom a single image to be present together as evaluation criteria or toset individual weights for how images are close to each other (degreesof coincidence) and the evaluation value from the single image.Moreover, it is also possible to individually set weights fordifferences in regions of interest. Also, a weight between images forevaluation (sample images), a weight in a single image evaluationscheme, weights of regions of interest, and a weight of a balancebetween maximization and optimization can be independently adjusted, andit is possible to allow the user to easily perform adjustment.

[13] Also, an example of an inspection method according to the presentdisclosure is a method for inspecting a workpiece using an inspectiondevice that includes an image-capturing unit, an image area settingunit, an optimization calculation unit, and a determination unit, themethod including: by the image-capturing unit, imaging at least oneworkpiece illuminated in a plurality of predetermined illumination lightemission patterns to acquire a plurality of evaluation workpiece images,each of which is associated with each predetermined illumination lightemission pattern and each workpiece; by the image area setting unit,setting a plurality of image areas associated with a plurality ofdifferent labels that indicate non-defective products or defectiveproducts for the evaluation workpiece images; by the optimizationcalculation unit, generating a first index, an output value of whichincreases as differences between image areas associated with labelsindicating the non-defective products and image areas associated withlabels indicating the defective products increase, generating a secondindex, an output value of which increases as differences between theimage areas associated with the labels indicating the non-defectiveproducts increase or as contrast in the image areas associated with thelabels indicating the non-defective products increases, from the imageareas, and calculating an illumination light emission pattern forinspection such that (an evaluation value of) the first index becomeslarger and (an evaluation value of) the second index becomes smaller;and by the determination unit, performing image processing on aninspection workpiece image obtained by the image-capturing unit imaginga workpiece to be inspected illuminated in the illumination lightemission pattern for inspection to determine a pass/fail label of theworkpiece to be inspected. With this configuration, effects andadvantages equivalent to those in the example of the inspection deviceaccording to the present disclosure described above in [1] are achieved.

[14] Also, a control program according to an example of the presentdisclosure causes a computer to execute the imaging of at least oneworkpiece, the setting of the image areas, the calculating of theillumination light emission pattern for the optimization, and thedetermining of the pass/fail label.

[15] Also, an example of a recording medium according to the presentdisclosure is a computer-readable non-transitory recording medium thatrecords a control program for causing a computer to execute the imagingof at least one workpiece, the setting of the image areas, and thecalculating of the illumination light emission pattern for theoptimization, and the determining of the pass/fail label.

Note that in the present disclosure, “units” and “devices” do not simplymean physical means but include configurations realizing functions thatthe “units” and the “devices” have using software. Also, functions thatone “unit” or one “device” has may be realized by two or more physicalmeans or devices, or alternatively, functions of two or more “units” or“devices” may be realized by one physical means or device. Further,“units” and “devices” are concepts that can be stated as “means” and“systems”, for example, instead.

Advantageous Effects of Invention

According to the present disclosure, image areas in appropriate regionsof interest are set from evaluation workpiece images, and a first indexindicating variations in workpieces belonging to different labels (forexample, variations between non-defective products and defectiveproducts) and a second index indicating variations in workpiecesbelonging to the same label (for example, variations in a non-defectiveproduct or variations in individual products) are set. Then, it ispossible to calculate an illumination parameter vector as an optimizedsolution through optimization using both the first index and the secondindex and to determine the label of the workpiece to be inspected usingan optimized illumination light emission pattern obtained on the basisof the illumination parameter vector. It is thus possible to emphasizedefects such as scratches in the workpieces (to increase differencesbetween non-defective products and defects to allow the defects to beeasily recognized), for example, and to curb influences of randomvariations in a non-defective product or variations in individualproducts caused by the background image of the workpieces (to reducevariations in a non-defective product). In this manner, it is possibleto reduce missing of defects in workpieces and determination errors andto realize improvement in inspection performance and inspectionefficiency of workpieces. Also, it is possible to improve a processingspeed (throughput) in the entire system including the inspection device,to save the storage capacity, to reduce the amount of data to becommunicated, and to enhance reliability of inspection through suchefficient inspection processing.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a plan view (partial perspective view) schematicallyillustrating an example (system configuration example) of an applicationsituation of an inspection device according to an embodiment of thepresent disclosure.

FIG. 1B is a plan view schematically illustrating an example ofdisposition of a light source used in the inspection device according tothe embodiment of the present disclosure.

FIG. 2 is a plan view schematically illustrating a hardwareconfiguration of a control device included in the inspection deviceaccording to the embodiment of the present disclosure.

FIG. 3 is a plan view schematically illustrating an example of afunctional configuration of the inspection device according to theembodiment of the present disclosure.

FIG. 4 is a schematic view illustrating an image synthesis model on theassumption of establishment of linearity in single-shot inspection usingmulti-channel illumination according to the present disclosure.

FIG. 5 is a schematic view illustrating an image synthesis model on theassumption of establishment of linearity in multi-shot inspection usingmulti-channel illumination according to the present disclosure.

(A) of FIG. 6 and (B) of FIG. 6 are flowcharts illustrating an exampleof processing procedures in an illumination optimization stage and aninspection stage according to the embodiment of the present disclosure,respectively.

FIG. 7 is a schematic view illustrating an overview of a procedure foracquiring an inspection workpiece image according to a thirdmodification example.

(A) of FIG. 8 and (B) of FIG. 8 are flowcharts illustrating an exampleof processing procedures in an illumination optimization stage and aninspection stage according to a fourth modification example,respectively.

(A) of FIG. 9 and (B) of FIG. 9 are flowcharts illustrating an exampleof processing procedures in an illumination optimization stage and aninspection stage according to a sixth modification example,respectively.

(A) of FIG. 10 and (B) of FIG. 10 are flowcharts illustrating an exampleof processing procedures in an illumination optimization stage and aninspection stage according to a seventh modification example,respectively.

(A) of FIG. 11 and (B) of FIG. 11 are schematic plan views illustratingan example of a display screen of a user interface in an example of theinspection device according to the embodiment of the present disclosure.

FIG. 12 is a schematic plan view illustrating another example of thedisplay screen of the user interface in an example of the inspectiondevice according to the embodiment of the present disclosure.

(A) of FIG. 13 to (C) of FIG. 13 are photographs (reconstructed images)illustrating an example of an illumination optimization result obtainedusing the Fisher's linear discrimination performed by the inspectiondevice according to the present disclosure.

FIG. 14 is a photograph illustrating an example of an illuminationoptimization result obtained through the Fisher's linear discriminationusing an evaluation function of a ratio type (Db/Dw) in the inspectiondevice according to the present disclosure.

FIG. 15 is a photograph illustrating an example of an illuminationoptimization result obtained through the Fisher's linear discriminationusing an evaluation function of a difference type (Db−λ·Dw) in theinspection device according to the present disclosure.

(A) of FIG. 16 to (C) of FIG. 16 are schematic plan views illustratingexamples of the display screen of the user interface according to thesixth modification example and the seventh modification example.

(A) of FIG. 17 and (B) of FIG. 17 are schematic plan views illustratingexamples of the display screen of the user interface according to aneighth modification example.

(A) of FIG. 18 illustrates an example of a pop-up window PW fordisplaying a result of setting a weight (aforementioned δ_(between)) ofa first index Db and a weight (aforementioned δ_(within)) of a secondindex Dw, which are evaluation criteria, and performing illuminationoptimization according to a ninth modification example. (B) of FIG. 18is a flowchart illustrating an example of a processing procedureaccording to the ninth modification example, and (C) of FIG. 18 is aschematic plan view illustrating an example of a result selected in theninth modification example.

(A) of FIG. 19 and (B) of FIG. 19 are schematic plan views illustratingexamples of the display screen of the user interface according to atenth modification example.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment according to an example of the presentdisclosure will be described with reference to drawings. However, thepresent embodiment described below is merely an example and is notintended to exclude applications of various modifications or techniquesthat will not be explicitly described below. In other words, an exampleof the present disclosure can be performed with various modificationswithout departing from the gist thereof. Also, the same or similarreference signs will be applied to the same or similar components in thefollowing description regarding drawings, and the drawings areschematically illustrated and do not necessarily conform to actualdimensions, ratios, and the like. Further, mutual relationships andratios of dimensions in the drawings may be included. Also, it is amatter of course that the present embodiment described below is a partof embodiments of the present disclosure and is not all the embodimentsthereof. Further, all other embodiments obtained by those skilled in theart with no need of creative actions are included in the protectionscope of the present disclosure on the basis of the present embodimentof the present disclosure.

§ 1 Application Example

In the present disclosure, a workpiece to be inspected (product to beinspected) is illuminated with a light source that has variableillumination light emission parameters and has desired illuminationlight emission patterns, and the workpiece is imaged with an appropriatesensor to acquire an inspection workpiece image. Then, inspection of theworkpiece (for example, label determination depending on whether or notthere are any defects such as scratches) is performed through imageprocessing on the inspection workpiece image. The illumination lightemission parameters include, for example, a light emission position, alight emission intensity, and a chromaticity of the light source and canbe represented as a single vector or a plurality of vectors (matrix).Also, the illumination light emission pattern in the inspection isoptimized prior to the acquisition of the inspection workpiece image.

In the optimization, a workpiece for evaluation illuminated with each ofa plurality of predetermined illumination light emission patterns of thelight source is imaged to obtain a plurality of (patterns of) evaluationworkpiece images (sample images). Then, for a combination of a pluralityof different labels (classes, categories) and at least one region ofinterest (RIO) with a predetermined shape, images in the region ofinterest in a plurality of evaluation workpiece images are extracted toset image areas (or a group thereof).

Then, a first index indicating variations in workpieces belonging todifferent labels (for examples, variations between non-defectiveproducts and defective products) is set from the image areas on thebasis of varying components in images belonging to the different labels.Also, a second index indicating variations in workpieces belonging tothe same label (for example, variations in a non-defective product orvariations in individual products) is set from the image areas on thebasis of varying components in images belonging to the same label. Next,as illumination optimization, an illumination parameter vector as anoptimized solution that maximizes the first index when at least oneworkpiece is illuminated and imaged in an illumination light emissionpattern for evaluation corresponding to an arbitrary combination of aplurality of predetermined illumination light emission patterns andminimizes the second index is obtained. Then, the workpiece to beinspected is inspected using an optimized illumination light emissionpattern obtained on the basis of the illumination parameter vector.

Here, an example of a situation to which an example of the presentdisclosure is applied will be described using FIGS. 1A and 1B, first.FIG. 1A is a plan view (partial perspective view) schematicallyillustrating an example (system configuration example) of a situation towhich the inspection device according to an embodiment of the presentdisclosure is applied. Also, FIG. 1B is a plan view schematicallyillustrating an example of disposition of a light source used in theinspection device according to the present embodiment of the presentdisclosure.

An inspection system 1 performs appearance inspection of a workpiece 4through execution of image analysis processing on an input imageobtained by imaging the workpiece 4 that is transported on a beltconveyor 2 and that is to be inspected, for example. Although as atypical example of the image analysis processing, inspection regardingwhether or not there are defects on the surface of the workpiece 4 willbe described as an application example, the present disclosure is notlimited thereto and can be applied to identification of types of defectsor measurement of outer shapes, for example.

A camera 102 that serves as a sensor including a light source LSintegrated therewith is disposed above the belt conveyor 2 and isconfigured such that an imaging field of view 6 of the camera 102includes a predetermined area on the belt conveyor 2. Here, amulti-channel illumination such as a multi-direction multi-color (MDMC)illumination, for example, is exemplified as the light source LS, andmore specifically, it is possible to exemplify the illuminationdescribed in Japanese Patent Application No. 2018-031747 of the presentapplicant, for example. The light source LS that is a multi-channelillumination has a plurality of channel illuminations LSi. Morespecifically, as illustrated in FIG. 1B, the light source LS has onechannel illumination LSi with a circular shape in a plan view and twelvechannel illuminations LSi that form a fan shape disposed coaxiallyaround the one channel illumination LSi. In this case, the light sourceLS is configured of a total of thirteen channel illuminations LSi, andin a case in which each of the channel illuminations LSi emits light ofthree colors, 13×3=39 predetermined illumination light emission patternsof a single color and a single channel are obtained. Further, data ofevaluation workpiece images generated through imaging performed by thecamera 102 is transmitted to a control device 100. The imaging performedby the camera 102 is executed periodically or as an event.

The control device 100 controls the illumination light emission patternsof the light source LS on the basis of predetermined illuminationparameters and calculates an optimized illumination light emissionpattern for inspecting the workpiece 4 to be inspected, using theevaluation workpiece images and the illumination light emission patternfor evaluation. Also, the control device 100 includes a learning machinethat has a convolutional neural network (CNN) engine for appearanceinspection of the workpiece 4. A feature detection image for each classis generated from the input image using the CNN engine. Whether or notthere are defects in the workpiece 4 to be inspected (labeldetermination) is determined on the basis of the generated one or aplurality of feature detection images, or it is also possible to detectthe sizes, the positions, and the like of the defects.

The control device 100 is connected to a programmable controller (PLC)10, a database device 12, and the like via an upper network 8.Arithmetic operation results and detection results obtained by thecontrol device 100 may be transmitted to the PLC 10 and/or the databasedevice 12. Note that in addition to the PLC 10 and the database device12, arbitrary devices may be connected to the upper network 8. Also, adisplay 104 that serves as an output unit for displaying a state duringprocessing, detection results, and the like and a keyboard 106 and amouse 108, for example, that serve as input units for receiving user'soperations may be connected to the control device 100.

§ 2 Configuration Example Configuration Example

Next, an example of a hardware configuration of the control device 100included in the inspection system 1 that is an inspection deviceaccording to an embodiment of the present disclosure will be describedusing FIG. 2. FIG. 2 is a plan view schematically illustrating thehardware configuration of the control device 100.

The control device 100 may be realized using a general-purpose computerconfigured in accordance with a general-purpose computer architecture inone example. The control device 100 includes a processor 110, a mainmemory 112, a camera interface 114, an input interface 116, a displayinterface 118, a communication interface 120, and a storage 130. Thesecomponents are typically connected via an internal bus 122 such that thecomponents can communicate with each other.

The processor 110 realizes functions and processing, which will bedescribed later, by developing and executing, in the main memory 112,various programs stored in the storage 130. The main memory 112 isconfigured of a volatile memory and functions as a work memory needed bythe processor 110 to execute programs.

The camera interface 114 is connected to the camera 102 and acquires aninput image 138 captured by the camera 102. The camera interface 114 mayprovide an instruction for an imaging timing or the like to the camera102.

The input interface 116 is connected to the input units such as thekeyboard 106 and the mouse 108 and acquires commands indicatingoperations and the like performed by the user on the input units.

The display interface 118 outputs, to the display 104, variousprocessing results generated by the processor 110 through execution ofprograms.

The communication interface 120 is in charge of processing forcommunication with the PLC 10, the database device 12, and the like viathe upper network 8.

The storage 130 stores programs for causing a computer to function asthe control device 100, such as a program for various kinds ofprocessing (illumination optimization and image processing) 132including an operating system (OS). The storage 130 may further storeillumination parameters 134 for realizing the illumination optimizationprocessing, a learning machine parameter 136 for realizing the imageprocessing, the input image 138 acquired from the camera 102, and anestimated image 140 that is a measurement result obtained from the inputimage 138. The illumination parameters 134 can include, for example,predetermined illumination light emission patterns of the light sourceLS used in an illumination optimization stage (in a case of MDMCillumination, a plurality of illumination light emission patterns with abasic intensity of each channel illumination LSi of the light sourceLS), an illumination pattern for evaluation, an illumination parametervector and an illumination parameter matrix corresponding to theoptimized illumination light emission pattern for inspection, and thelike. Also, the learning machine parameters 136 can include illuminationparameters, inspection algorithm parameters, and the like used byvarious machine learning models in a learning stage and an inspectionstage, for example.

The program for various kinds of processing 132 stored in the storage130 may be installed in the control device 100 via an optical recordingmedium such as a digital versatile disc (DVD) or a semiconductorrecording medium such as a universal serial bus (USB) memory.Alternatively, the program for various kinds of processing 132 may bedownloaded from a server device or the like on a network.

In such a case in which a general-purpose computer is used forimplementation, some of the functions according to the presentembodiment may be realized by calling and performing necessary softwaremodules from among software modules provided by the OS in apredetermined order and/or at a predetermined timing. In other words,the program for various kinds of processing 132 according to the presentembodiment may not include all the software modules for realizing thefunctions according to the present embodiment and may provide necessaryfunctions in cooperation with the OS.

Also, the program for various kinds of processing 132 may be providedwhile being incorporated in another program as a part thereof. In such acase, the program for various kinds of processing 132 itself does notinclude a module included in another program that can be combinedtherewith as described above, and processing is executed in cooperationwith that other program. In this manner, a form in which the program forvarious kinds of processing 132 according to the present embodiment isincorporated in another program may employed.

Note that although FIG. 2 illustrates the example in which the controldevice 100 is realized using a general-purpose computer, the presentdisclosure is not limited thereto, and all or some functions thereof maybe realized using a dedicated circuit (for example, an applicationspecific integrated circuit (ASIC) or a field programmable gate array(FPGA)). Further, an external device connected to the network may becaused to be in charge of a part of the processing.

As described above, the inspection system 1 corresponds to an example ofthe “inspection device” according to the present disclosure in terms ofthe hardware configuration. Also, the workpiece 4 corresponds to anexample of the “workpiece” according to the present disclosure. Further,the light source LS corresponds to an example of the “light source”according to the present disclosure, and the camera 102 corresponds toan example of the “sensor” according to the present disclosure.

[Functional Configuration]

Next, an example of a functional configuration of the inspection system1 that is the inspection device according to an embodiment of thepresent disclosure will be described using FIG. 3. FIG. 3 is a plan viewschematically illustrating an example of the functional configuration(functional module) of the inspection system 1. As illustrated in FIG.3, the control device 100 of the inspection system 1 can include animage-capturing unit 141, an image area setting unit 142, anoptimization calculation unit 143, a determination unit 144, and astorage unit 145.

The image-capturing unit 141, the image area setting unit 142, theoptimization calculation unit 143, and the determination unit 144 in thecontrol device 100 can be realized by a general-purpose processor butare not limited thereto in the present disclosure, and some or allfunctions of these components may be realized using a dedicated circuit(for example, an application specific integrated circuit (ASIC) or afield programmable gate array (FPGA)). Further, an external deviceconnected to the network may be caused to be in charge of a part ofprocessing.

The image-capturing unit 141 illuminates and images at least oneworkpiece 4 including a workpiece 4 that has an area corresponding to aplurality of different labels (classes, categories) in a plurality ofpredetermined illumination light emission patterns with the light sourceLS and the camera 102 in the illumination optimization stage (teachingphase). Also, the image-capturing unit 141 illuminates and images theworkpiece to be inspected in an optimized illumination light emissionpattern with the light source LS and the camera 102 in the learningstage and the inspection stage.

The image area setting unit 142 extracts, for a combination of aplurality of different labels and at least one region of interest with apredetermined shape (for example, a rectangular shape, a circular shape,or an oval shape), images in the region of interest (ROI) in a pluralityof evaluation workpiece images (input images 138 for evaluation) andsets them as image areas for evaluation, in the illuminationoptimization stage.

The optimization calculation unit 143 sets the first index indicatingvariations in workpieces belonging to different labels on the basis ofvarying components (for example, differences or dispersion) in imagesbelonging to the different labels (classes) from the image areas. Also,the optimization calculation unit 143 sets the second index indicatingvariations in workpieces belonging to the same label on the basis ofvarying components in images belonging to the same label from the imageareas. Then, the optimization calculation unit 143 obtains anillumination parameter vector as an optimized solution that maximizes(an evaluation value of) the first index when at least one workpiece isilluminated and imaged in an illumination light emission pattern forevaluation corresponding to an arbitrary combination of a plurality ofpredetermined illumination light emission patterns of the light sourceLS and minimizes (an evaluation value of) the second index. Then, anoptimized illumination light emission pattern is calculated on the basisof the illumination parameter vector.

The determination unit 144 determines a label of the workpiece to beinspected through image processing of the inspection workpiece image(the input image 138 for inspection) obtained by the image-capturingunit 141 and the estimated image 140 in the inspection stage. Note thatthe determination unit 144 can perform inspection of the workpiece 4 onthe belt conveyor 2 using a trained machine learning model, for example,and outputs a final inspection result related to the workpiece 4. In acase in which a learning machine used for learning is a convolutionalneural network (CNN) that generates features extracted from an image,for example, the determination unit 144 can include a determinationmachine that generates a final inspection result through an applicationof determination criteria to the features extracted by the learningmachine.

The storage unit 145 stores a plurality of evaluation workpiece images(input images 138 for evaluation) which associated with thepredetermined illumination light emission patterns and the workpiece 4and which are obtained by the image-capturing unit 141 in theillumination optimization stage and the inspection workpiece image(input image 138 for inspection) obtained by the image-capturing unit141 in the learning stage and the inspection stage. Also, the storageunit 145 stores various arithmetic operation parameters obtained inarithmetic operation processing performed by the optimizationcalculation unit 143 and the optimized illumination parameter vector forinspection. Further, the storage unit 145 is realized by theaforementioned storage 130 that stores programs or data necessary foroperations of the inspection system 1. Note that the control device 100may not include the storage unit 145 and may use an external (device)storage instead of the storage unit 145.

As described above, the control device 100 corresponds to an example ofthe “image-capturing unit” according to the present disclosure alongwith the light source LS and the camera 102, in terms of a functionalmodule. Also, the control device 100 functions as an example of the“image area setting unit”, the “optimization calculation unit”, and the“determination unit” according to the present disclosure.

§ 3 Operation Example (Evaluation Criteria for IlluminationOptimization)

Here, an overview of importance of illumination optimization andevaluation criteria thereof will be described, and then a specificprocessing procedure will be described with reference to drawings andthe like. First, a purpose of illumination design in an appearanceinspection of a workpiece and the like is to allow accuratediscrimination between non-defective products (normal products) anddefective products (products with defects). Although this problem issolved by manual operations of skilled persons in the related art, it isnecessary to determine and limit “determination criteria” and “a degreeof freedom in control of illumination design” in advance in order tosystematically solve the problem.

Here, an illumination optimization problem in a case in which aprescribed discrimination algorithm is provided as a determinationcriterion and a multi-channel illumination such as MDMC illumination isprovided as a degree of freedom in control is formulated as a “crossentropy minimization problem” that causes categorization betweennon-defective products and defective products to conform to a correctanswer. Note that in a case in which a discrimination machine in thelabel determination of the workpiece is a machine learning machine, itis also possible to perform the illumination optimization and thelearning performed by the discrimination machine at the same time and tocause tuning for exhibiting the best performance in both theillumination optimization and the learning to be automaticallyperformed.

However, what becomes a big problem when cross entropy minimization isto be performed is that a large number of samples with the non-defectiveproduct/defective product labels applied thereto are needed. In a casein which illumination with a particularly large degree of freedom is tobe optimized, it is difficult to uniquely determine optimal illuminationwith criterion that a small number of non-defective product/defectiveproduct labels are merely distinguished. Such a problem is a big problemthat is to be considered at the time of activating an inspection devicewith which many evaluation workpiece images (sample images) cannot beobtained, in particular.

Thus, the present disclosure will propose a method for optimizingillumination on the basis of evaluation criteria (contrast, brightness,closeness) of an image itself in order to solve the problem. In thiscase, it is possible to roughly exemplify the following two requirementsas requirements required for optimized illumination design in theappearance inspection of a workpiece or the like.

[Requirement 1] To cause features to be easily recognizable for easyidentification of a non-defective product (label) and a defectiveproduct (label) (that is, to cause defects to be easily recognized)[Requirement 2] To cause variations in a non-defective product(variations in individual products) to be less easily recognizable

However, since both the requirements are typically conflictingcharacteristics, performing of illumination design with a balancetherebetween is a difficult problem in the illumination optimization. Inorder to quantitatively examine such a requirement here, a difference inimages between non-defective products and defective products will bedescribed as Db (first index: variations between non-defective productsand defective products), and a difference in images of a non-defectiveproduct will be described as Dw (second index: variations in anon-defective product). Although these may be defined using arbitraryevaluation criteria, a quadratic function is used in the embodiment andmodification examples of the present disclosure (that is, variationswill be defined as dispersion). In the illumination design that satisfyboth the aforementioned requirements, it is necessary to solve anoptimization problem that increases the value of Db and decreases thevalue of Dw at the same time. In this case, although it is necessary todetermine a balance therebetween in order to optimize the two evaluationfunctions Db and Dw at the same time, an example in which evaluationexpressions of a ratio type (Db/Dw) and a difference type (Db−λ·Dw) areused will be described in the present disclosure.

Note that if a definition that it is preferable to increase the value ofDb and to decrease the value of Dw is employed, arbitrary evaluationcriteria may be used. In a case in which the ratio type (Db/Dw) is used,there is a feature that no problem occurs in optimization even if adifference between the values of Db and Dw is large to some extent. Notethat there may be a case in which an absolute value of an illuminationintensity is not fixed and a standardized solution is obtained due tothe ratio type. On the other hand, in a case of the difference type(Db−λ·Dw), a problem for determining an absolute value of the solutionis easily handled. Note that in a case in which the difference betweenthe values of Db and Dw is large to some extent, it is possible toaddress the problem by introducing a parameter (λ) for adjustment toachieve a balance therebetween.

Further, although it is necessary to obtain a solution of theoptimization problem under a restriction condition that a light emissionintensity of illumination (an intensity of each channel of themulti-channel illumination) is a non-negative value regardless of whichevaluation criteria is to be used, this is typically difficult.Generally, it is possible to obtain the solution at a high speed if asecondary planning method or the like is used in a case of a convexquadratic function. However, even if both Db and Dw are simple convexquadratic functions, a quadratic coefficient is not a positive definiteif a difference therebetween is used, and a so-called nonconvexquadratic programming (QP) problem occurs. Similarly, a so-callednonconvex quadratically constrained quadratic programming (QCQP) problemoccurs in a case in which the evaluation function of the ratio type isused. In order to solve these optimization problems, an optimizationmethod based on semidefinite programming (SDP) will be referred to inthe embodiment of the present disclosure.

Note that meanings of each sign used in expressions in the followingdescription of the embodiment and each modification example are as shownin Table 1. Bold lowercases represent vectors, and bold uppercasesrepresent matrixes (these will be described in a non-bold manner in thesentences). The other signs are scalars.

Meaning Sign Number of channels in multi-channel illumination L Numberof workpiece images captured with illumination light emission N patternfor evaluation (for teaching) changed Number of comparison target groupsin region of interest (ROI) P Number of labels (classes, categories) KNumber of images captured in multi-shot inspection M Illumination lightemission pattern for evaluation (teaching) h_(i) (1 ≤ i ≤ N) H(horizontally aligned) Workpiece captured image at h_(i) f_(i) (1 ≤ i ≤N) f (vertically aligned) Group of f S_(p) ^((k)) Matrix expression oflinear image processing in inspection stage B Image synthesis weight(single shot) x_(i) (1 ≤ i ≤ N) x (aligned in a vertical vector) Imagesynthesis weight (multiple shots) w_(i, j) (1 ≤ i ≤ N, 1 ≤ j ≤ M) (*expressed as W to avoid confusion with X) W (aligned in a matrix)Optimization variable of SDP X Optimized illumination intensity to befinally obtained u_(i) (1 ≤ i ≤ L) (single shot) u (aligned in avertical vector) Optimized illumination intensity to be finally obtainedu_(i) (1 ≤ i ≤ M) (multiple shots) U (horizontally aligned)

(Equivalency of Imaging in Multi-Channel Illumination and Inner ProductArithmetic Operation)

Although the image-capturing unit 141 using a multi-channel illuminationas the light source LS is used in the inspection system 1 according tothe present disclosure, it is assumed here linearity of the inspectionsystem 1 on the side of the sensor (linearity between luminance and a QLvalue) is established. This means that linearity is established in asystem including all kinds of image correction such as signal processingincluding color filter demosaicing, gamma correction, and dark currentoffset correction. In order to avoid non-linearity due to pixelsaturation, high dynamic range (HDR) synthesis or the like may be usedto capture an image. At this time, a light emission intensity of eachchannel illumination is denoted as a vector in Expression (1) below inconsideration of a situation in which a workpiece 4 illuminated with Lchannel illuminations LSi is imaged.

[Math. 1]

u=(u ₁ ,u ₂ , . . . u _(L))^(T) , u _(i)≥0  (1)

At this time, if an image obtained by aligning, as column vectors,images that are captured with only i-th illumination turned on with apredetermined intensity as represented by Expression (2) described belowand with the other illuminations turned off is defined as fi, a capturedimage g under an arbitrary illumination condition u can be modeledthrough image synthesis as represented by Expression (3) describedbelow.

[Math. 2]

u _(i)=1,u _(j)=0(j≠i)  (2)

g=Σ _(i=1) ^(L) u _(i) f _(i) =Af  (3)

Here, each of A and f is a multi-channel image (one large column vector;Expression (5)) in which a projection matrix defined by Expression (4)described below and N images are vertically aligned.

[Math. 3]

A=u ^(T) ⊗I  (4)

f=Σ _(i=1) ^(L) e _(i) ⊗f _(i)  (5)

Also, each operator in the above expressions is as follows.

[Math. 4]

⊗: indicates a Kronecker product

e_(i): indicates a standard basis of

^(L) ^(i)

I: indicates a unit matrix.

In this manner, optimization of multi-channel illumination is equivalentto generation of the feature vector g in an inner product arithmeticoperation from an original image f, that is, optimal design in a linearprojection direction u in the projection matrix A. Here, FIGS. 4 and 5are schematic views illustrating an image synthesis model on theassumption of establishment of linearity in single-shot inspection andmulti-shot inspection using the multi-channel illumination according tothe present disclosure.

Next, an example of an inspection method including an illuminationoptimization method according to an embodiment of the present disclosurewill be described with reference to (A) of FIG. 6 and (B) of FIG. 6. (A)of FIG. 6 and (B) of FIG. 6 are flowcharts illustrating an example ofprocessing procedures in the illumination optimization stage and theinspection stage according to the embodiment of the present disclosure.Note that the processing procedures described below are merely examples,and the processing may be changed as long as possible within the scopeof the technical idea of the present disclosure. In the processingprocedures described below, omission, replacement, and addition of stepscan be appropriately made in accordance with the embodiment and eachconfiguration example.

(Step S410)

In Step S410 (imaging step), the camera 102 and the image-capturing unit141 illuminate and image (C) workpieces 4 for evaluation in each of aplurality of predetermined illumination light emission patterns (Npatterns) to acquire C×N evaluation workpiece images in an illuminationoptimization stage in single-shot inspection.

Here, a situation in which one workpiece 4 is imaged in N predeterminedillumination light emission patterns to determine (teach) an optimalillumination light emission pattern in a situation in which L channelilluminations are mounted in a multi-channel illumination is generallyconsidered. First, an illumination light emission pattern in n-th(1≤n≤N) imaging is defined by Expressions (6) and (7) described below.

[Math. 5]

h _(n)=(h _(1,n) ,h _(2,n) , . . . ,h _(L,n))^(l) , h _(i,n)≥0  (6)

H=[h ₁ ,h ₂ , . . . ,h _(N)]  (7)

It is possible to reconfigure a captured image of a workpiece in a casein which the workpiece is illuminated in an arbitrary light emissionpattern by imaging the workpiece 4 in these predetermined illuminationlight emission pattern. This is equivalent to performing of estimationof a light transport (LT) matrix. In order to acquire all degrees offreedom of the LT matrix, it is generally desirable to determine anillumination light emission pattern to satisfy the relationship ofExpression (8) described below, that is, to achieve linear independence.

[Math. 6]

rank H=min(N,L)  (8)

Here, in order to fully use all the degrees of freedom of Lilluminations, at least N=L is to be satisfied, and at that time, H hasto be a full rank. On the other hand, in a case in which N<L, this is aneffective method for shortening an imaging time of the workpiece 4though it is not possible to sufficiently use the degrees of freedom ofthe number of illuminations. Although various methods to realize thisare conceivable, representative examples thereof include a method ofperforming LT matrix estimation using compressed sensing. On the hand, acase in which N>L is also conceivable, and this may lead to anunnecessary large number of captured images that exceed a necessarynumber in terms of sufficient utilization of the degrees of freedom ofthe illuminations. However, this can be selected for another purpose ofenhancing an SN ratio or widening a dynamic range.

An image obtained by aligning, as column vectors, the evaluationworkpiece images captured in these predetermined illumination lightemission patterns is defined as fi, and a multi-channel image vectorobtained by vertically aligning and organizing N (N patterns of) fi isdefined as Expression (9) described below. This is equivalent to L=N inExpression (5) described above.

[Math. 7]

f=Σ _(i=1) ^(N) e _(i) ⊗f _(i)  (9)

In the above expression, each arithmetic operation is as follows.

[Math. 8]

⊗: indicates a Kronecker product

e_(i): indicates a standard basis of

^(N)

(Steps S412 and S414)

Next, in Step S412 (image area setting step), the image area settingunit 142 extracts, for a combination of a plurality of different labelsand at least one region of interest (ROI) with a predetermined shape(for example, a rectangular shape, a circular shape, or an oval shape),images in the predetermined region of interest from the evaluationworkpiece images acquired in Step S410 on the basis of a user'sinstruction, for example, and sets image areas. In Step S414(calculation step), the optimization calculation unit 143 obtains,through discriminant analysis such as the Fisher's lineardiscrimination, for example, an illumination parameter vector as anoptimized solution that maximizes the first index Db (variations betweennon-defective products and defective products; how easily defects arerecognizable) when the workpiece 4 is illuminated and imaged in anarbitrary illumination light emission pattern for evaluation andminimizes the second index Dw (variations in a non-defective product;variations in individual products), using image data in the image areas.

Here, (A) of FIG. 11 and (B) of FIG. 11 are schematic plan viewsillustrating examples of a display screen of a user interface in anexample of the inspection device according to the embodiment of thepresent disclosure. In (A) of FIG. 11, an image in a region of interest(ROI) R0 designated by the user and an index Ind (including a weight;see the ninth modification example described below for the weight) ofthe region of interest R0 are displayed. In order to set the region ofinterest R0, it is possible to use a pop-up window PW for choosing aregion of interest illustrated in (B) of FIG. 11, for example. A list ofevaluation workpiece images captured in advance in the predeterminedillumination light emission patterns is displayed in the pop-up windowPW, and a desired evaluation workpiece image is displayed in an enlargedmanner by the user performing an operation such as tapping on the image(in the illustrated example, the second workpiece image has beenselected).

The user can extract a region in a desired range as the region ofinterest R0 by performing a tapping or pinching operation on the desiredregion in the enlarged display image. The image in the thus selectedregion of interest R0 is pop-up displayed as illustrated in (A) of FIG.11. Also, the pop-up window PW illustrated in (B) of FIG. 11 may bedisplayed again through tapping of a touch button Bt displayed in (A) ofFIG. 11 such that the region of interest R0 can be changed or anotherregion of interest R0 can additionally be selected. Further, a pluralityof regions of interest R0 may be selected at a time from one evaluationworkpiece image. In a case in which a plurality of different regions ofinterest R0 are selected in the same area (same p in the index Ind), itis desirable to choose the regions such that the shapes and the sizes ofthe regions are the same.

(Arbitrary Illumination Light Emission Pattern)

Here, the arbitrary light emission pattern u within a range in whichestimation can be made from the N evaluation workpiece images and thecaptured image g at that time are expressed by Expression (10) to (12)described below.

[Math. 9]

u=Hx, x=(x ₁ ,x ₂ , . . . ,x _(N))^(T)  (10)

g=Σ _(i=1) ^(N) x _(i) f _(i) =Af  (11)

A=x ^(T) ⊗I  (12)

Here, xi denotes an image synthesis weight of the captured images in thepredetermined illumination light emission patterns. Note that a commandvalue of a light emission intensity of each channel illumination LSi inthe light source LS is calculated on the basis of the weight value butis not the weight value itself. Also, although this is generally allowedto be a negative value, the condition of Expression (13) described belowis applied since the light emission intensity of illumination has to bealways non-negative.

[Math. 10]

Hx≥0  (13)

Here, inequality signs for vectors and matrixes are assumed to beinequality signs for all elements. In a case in which H=I, that is, L=Nimages are captured with the channel illuminations LSi of the lightsource LS turned on one by one, u is equivalent to x. Therefore, it ispossible to obtain the optimal illumination parameter vector u at thattime by obtaining the optimal image synthesis weight x.

(Fisher's Linear Discrimination)

The Fisher's linear discrimination is a method by which it is possibleto obtain a projection vector that “maximizes inter-label (class)dispersion” and “minimizes intra-label (class)”. According to theFisher's linear discrimination, an arithmetic operation for generating afeature vector is a simple inner product, and the effect thereof is thussignificantly limited as compared with a method of employing variouslinear arithmetic operations used in recent years. However, in theoptimization of illumination parameters handled in the presentdisclosure, if imaging in the multi-channel illumination is regarded asan arithmetic operation as described above, the imaging ismathematically a simple inner product arithmetic operation and linearprojection themselves. Therefore, it is possible to state that theFisher's linear discrimination capable of obtaining optimal linearprojection is effective for the illumination optimization problem of thepresent disclosure.

Hereinafter, a problem of defining inter-label (class, category)dispersion and intra-label dispersion by focusing on how close imagesare (similarity) and calculating an illumination light emission patternof the multi-channel illumination that realizes the Fisher's lineardiscrimination based on the definition will be formulated. Generally, ina case in which workpieces are categorized into K labels (classes,categories), K=2 in discrimination only between non-defective products(PASS) and defective products (FAIL), for example. Indexes of the labelsare assumed to meet 1≤k≤K.

Also, it is assumed that there are P (≥1) groups as comparison targetsin the region of interests (ROI) of the evaluation workpiece images. Theillumination optimization criteria are defined depending on how largevariations in a group of images belonging to the same group are. Indexesof the comparison target groups in the region of interest are assumed tomeet 1≤p≤P.

At this time, it is assumed that image areas obtained by extractingimages at the portion of the region of interest from the evaluationworkpiece images are provided by Expression (14) described below foreach combination of k and p as described above prior to the illuminationoptimization.

[Math. 11]

S _(p) ^((k))  (14)

Also, the number C of the workpieces 4 for evaluation does notnecessarily conform to the number of elements of the image areas. Inother words, this is because in addition to the presence of a case inwhich a plurality of regions of interest are picked up from oneevaluation workpiece image, which region of interest belongs to whichregion of interest comparison target group may be arbitrary selected andarbitrarily set by the user, and these elements thus change depending ona user's instruction.

Here, FIG. 12 is a schematic plan view illustrating another example ofthe display screen of the user interface in an example of the inspectiondevice according to the present embodiment of the present disclosure andis also a diagram illustrating a state in which the image areas ofExpression (14) described above are set. Note that a weight is includedas the index Ind of the image in each of regions of interest R0 of theimage areas, and the weight will be described in detail in the ninthmodification example described below.

FIG. 12 is an example of the pop-up window PW in which an image areabelonging to an index p (area) of a comparison target group is displayedin a tab form. In the illustration, an image (the image at the left endin the illustrated paper surface) in the region of interest R0 chosen bythe user for each of the labels=1 and 2 (which correspond to anon-defective product label and a defective product label, respectively)belonging to an area=1 and images to each of which the user has applieda weight (index) are grouped. In the pop-up window PW in FIG. 12, theweight set for each area is also displayed, and in an image in theregion of interest R0 chosen by the user, dispersions (β and α,respectively) of the first index Db among labels and the second index Dwin the same label from an average are gauge-displayed. In addition, theweighted second index Dw(α) with the same label is displayed in eachweighted region of interest R0. Also, an addition button Bx for theregion of interest R0 (see FIG. 11 for selection of a region) isdisplayed beside each label in the pop-up window PW, and an additionbutton By for a label k is also displayed below the label.

The description will be repeated here. That is, it is assumed that Nevaluation workpiece images (Expression (15) described below) used inthe optimization stage (teaching) are to be captured each time byturning on the light source in accordance with (the light emissionpattern for evaluation) hn in the predetermined illumination lightemission pattern of each channel of the multi-channel illumination.Using these, an expectation value D_(within) of an intra-label (class)distance (variations in a non-defective product) and an expectationvalue D_(between) of an inter-label (class) distance (how easy defectsare recognizable) are defined as l2 distances as in Expressions (16) and(17) described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 12} \right\rbrack & \; \\{f \in \mathcal{S}_{p}^{(k)}} & (15) \\{{D_{within} = {\underset{k.p}{\mathbb{E}}{{Var}\left( {\mathcal{S}_{p}^{(k)},m_{p}^{(k)}} \right)}}},{m_{p}^{(k)} = {\underset{f \in \mathcal{S}_{p}^{(k)}}{\mathbb{E}}f}}} & (16) \\{{D_{between} = {\underset{p}{\mathbb{E}}{{Var}\left( {\mathcal{S}_{p}^{\prime},m_{p}^{\prime}} \right)}}},{\mathcal{S}_{p}^{\prime} = \left\{ m_{p}^{(k)} \right\}_{k}},{m_{p}^{\prime} = {\underset{k}{\mathbb{E}}m_{p}^{(k)}}}} & (17)\end{matrix}$

Here, Var (S, m) is calculated as represented by Expression (18)described below from a group S and an average m of the multi-channelimages, and this can be expressed as a secondary form of theillumination parameter vector x. The dispersion matrix at this time willbe described as Q.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 13} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {{\underset{f \in S}{\mathbb{E}}{{g - {\underset{f \in S}{\mathbb{E}}g}}}^{2}} = {\underset{f \in S}{\mathbb{E}}{{A\left( {f - {\underset{f \in S}{\mathbb{E}}f}} \right)}}^{2}}}} \\{= {{{tr}\mspace{11mu}{ARA}^{T}} = {{{tr}\mspace{11mu}\left( {x^{T} \otimes I} \right){R\left( {x \otimes I} \right)}} = {x^{T}{Qx}}}}}\end{matrix} & (18)\end{matrix}$

In Expression (18) described above, the matrixes R and Q arespecifically expressed as Expressions (19) and (20) described below.

$\begin{matrix}{\mspace{79mu}\left\lbrack {{Math}.\mspace{14mu} 14} \right\rbrack} & \; \\{R = {{{\underset{f \in S}{\mathbb{E}}\left( {f - {\underset{f \in S}{\mathbb{E}}f}} \right)}\left( {f - {\underset{f \in S}{\mathbb{E}}f}} \right)^{T}} = {\underset{f \in S}{\mathbb{E}}{\sum_{i,j}{e_{i}{e_{j}^{T} \otimes \left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)^{T}}}}}} & (19) \\{\mspace{79mu}{\lbrack Q\rbrack_{i,j} = {{\underset{f \in S}{\mathbb{E}}\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)}^{T}\left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}}} & (20)\end{matrix}$

In the expansion at and after this point, the dispersion matrixescorresponding to the expectation values D_(between) and D_(within) aredefined as dispersion matrixes Q_(between) and Q_(within), respectively,and an evaluation function is designed using these dispersion matrixes.Since the dispersion matrixes are in secondary forms related to theillumination parameter vector in this case unlike a typical case, therank of Q is not necessarily 1 even if the number of intra-label samplesis two, and there are almost full ranks in many cases though it dependson the configuration of the multi-channel illumination.

Then, in the Fisher's linear discriminant, x that maximizes theevaluation expression represented by Expression (21) described belowcorresponds to an image synthesis weight to be obtained, that is, anoptimization illumination parameter in the evaluation function of theratio type (Db/Dw) in order to realize maximization of the inter-label(class) dispersion and minimization of the intra-label (class)dispersion at the same time.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 15} \right\rbrack & \; \\{{{maximize}\mspace{11mu}\frac{x^{T}Q_{between}x}{x^{T}Q_{within}x}},\mspace{14mu}{{{subject}\mspace{14mu}{to}\text{:}\mspace{14mu}{x}^{2}} = 1}} & (21)\end{matrix}$

Note that the reason that the norm of x is restricted in the aboveexpression is because the norm becomes indefinite merely with theevaluation function. The optimal projection direction of the Fisher'slinear discrimination can be obtained as a maximum eigenvector of aneigenvalue in the general eigenvalue problem by a Lagrange undeterminedmultiplier method. However, what is problematic in the illuminationoptimization is a point that the eigenvector can be a negative value. Inother words, since it is not possible to actually realize a negativeillumination intensity although it is only necessary to set theillumination parameter vector in the direction that follows theeigenvector, a measure for solving this point will be described inmodification examples 5-2 and 5-3 described below.

Then, in Step S414, an illumination light emission pattern optimized forthe inspection is calculated on the basis of the illumination parametervector u corresponding to x that maximizes the evaluation expression(21) in the aforementioned Fisher's linear discrimination.

(Steps S420 and S422)

In Step S420, the camera 102 and the image-capturing unit 141 acquire aninspection workpiece image of a workpiece to be inspected using theillumination light emission pattern optimized for inspection in theinspection stage. Then, in Step S422, the determination unit 144performs label determination (for example, inspection regarding whetheror not there are defects such as scratches) of the workpiece.

§ 4 Effects and Advantages

As described above, according to an example of the inspection system 1that serves as the inspection device according to the present embodimentand the inspection method using the inspection system 1, it is possibleto obtain an optimized illumination light emission pattern capable ofcurbing influences of variations in non-defective product or individualvariations of the workpieces while emphasizing defects in workpieces. Asa result, it is possible to reduce missing of defects in workpieces anddetermination errors and thereby to improve inspection performance andinspection efficiency of the workpieces. Also, it is possible to reduceefforts and the number of processes required to optimize an illuminationparameter in the related art, thereby to improve producibility andeconomic efficiency, and further to contribute to saving of resourcessuch as storage resources.

Also, since the evaluation expression based on the evaluation functionof the ratio type (Db/Dw) is maximized using the Fisher's lineardiscrimination, it is possible to directly analyze the maximization ofDb and the minimization of Dw and to perform optimization even if thedifference between Db and Dw is large to some extent. Further, since theevaluation function of the ratio type is used, it is possible tosuitably obtain the optimized solution of any of the illuminationparameter in the single-shot inspection and the illumination parameterin the multi-shot inspection.

Here, (A) of FIG. 13 to (C) of FIG. 13 are photographs (reconstructedimages) illustrating examples of illumination optimization results usingthe Fisher's linear discrimination using the inspection device accordingto the present disclosure. (A) of FIG. 13 is an example of a comparison(Db) of a region of interest R1 with a defect (scratch HL on a hairline)and a region of interest R2 with no defect in the workpiece 4, and (B)of FIG. 13 is an example of comparison (Dw) between regions of interestR3 and R4 with no defects. (C) of FIG. 13 is an example of a result ofilluminating and imaging a portion of the scratch HL on the hairline inan illumination light emission pattern as the optimized solution thatmaximizes Db (variations between non-defective products and defectiveproducts; how easily defects are recognizable) and minimizes Dw(variations in a non-defective product or variations in individualproducts). As described above, it is possible to understand that thepresent disclosure can curb influences of variations in a non-defectiveproduct or variations in individual products of the workpieces whileemphasizing defects in the workpieces 4 and improve defect determinationperformance.

§ 5 Modification Examples

Although the embodiment as an example of the present disclosure has beendescribed above in detail, the above description is merely illustrationof an example of the present disclosure in all senses, it is a matter ofcourse that various improvements and modifications can be made withoutdeparting from the scope of the present disclosure, and the followingchanges can be made, for example. Note that in the followingdescription, similar reference signs will be used for components similarto those in the aforementioned embodiments, and description of pointssimilar to those in the aforementioned embodiment will be appropriatelyomitted. Also, the aforementioned embodiment and the followingmodification examples can be configured in appropriate combination.

5.1: First Modification Example

An inspection system according to a first modification example has aconfiguration similar to that of the inspection system 1 according tothe aforementioned embodiment other than that an evaluation expressionbased on an evaluation function of a difference type (Db−λ·Dw) is usedinstead of the evaluation expression based on the evaluation function ofthe ratio type (Db/Dw) in the Fisher's linear discrimination in theillumination optimization stage. In a case in which the evaluationfunction of the ratio type (Db/Dw) is replaced with the evaluationfunction of the difference type, the evaluation expression representedby Expression (22) described below is minimized by introducing aparameter λ (>0) for adjusting Db and Dw.

[Math. 16]

minimize: x ^(T) Q _(within) x−λx ^(T) Q _(between) x,

subject to: ∥x∥ ²=1,Hx≥0  (22)

In order to realize this, optimization based on SDP using dispersionmatrixes redefined as represented by Expressions (23) and (24) describedbelow may be performed similarly to the case of the ratio type (Db/Dw).However, it is necessary to optimally select the value of λ, and it ispossible to handle the case through manual parameter tuning.

[Math. 17]

Q′ _(within) =Q _(within) −λQ _(between)  (23)

Q′ _(between) =I  (24)

Here, FIG. 14 is a photograph illustrating an example of an illuminationoptimization result of the Fisher's linear discrimination using theevaluation function of the ratio type (Db/Dw) in the inspection deviceaccording to the present disclosure. Also, FIG. 15 is a photographillustrating an example of an illumination optimization result of theFisher's linear discrimination using the evaluation function of adifference type (Db−λ·Dw) in the inspection device according to thepresent disclosure. Both the photographs are workpiece images in aregion corresponding to the region of interest R1 with the defect (thescratch HL on the hairline) illustrated in (A) of FIG. 13 to (C) of FIG.13. In the case in which the evaluation function of the difference typeis used as illustrated in FIG. 15, it is possible to search for anoptimized solution by appropriately adjusting the parameter λ foradjusting Db and Dw to keep a balance.

According to the configuration of the first modification example, thereare advantages that it is possible to obtain an illumination parameterthat causes variations in non-defective product to be less easilyrecognizable and to cause defects to be easily recognizable and aproblem of determining an absolute value of a solution is easilyhandled, similarly to the ratio type. Further, it is possible to obtainthe optimized solution in a single shot, in particular, at a high speed.

5.2: Second Modification Example

As described above, the optimal projection direction of the Fisher'slinear discrimination can be obtained as a maximum eigenvector of aneigenvalue in a general eigenvalue problem by the Lagrange undeterminedmultiplier method. However, since it is not possible to actually realizea negative illumination intensity although the eigenvector can be anegative value in the illumination optimization, there may be a case inwhich handling of this point is effective.

On the other hand, the second modification example is an example inwhich the optimization calculation unit 14 includes a restrictioncondition that all components of the illumination parameter vector arenon-negative values in the Fisher's linear discrimination. Therefore, itis possible to obtain an optimized illumination parameter vectors, allcomponents of which are positive values, according to the modificationexample.

More specifically, a method of calculating an illumination optimalsolution under a non-negative condition at a high speed by amathematical programming method will be described here. An originalproblem is to maximize the evaluation expression of the Fisher's lineardiscrimination represented by Expression (25) described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 18} \right\rbrack & \; \\{{{maximize}\text{:}\mspace{14mu}\frac{x^{T}Q_{between}x}{x^{T}Q_{within}x}},} & (25) \\{{{{subject}\mspace{14mu}{to}\text{:}\mspace{14mu}{x}^{2}} = 1},{{Hx} \geq 0}} & \;\end{matrix}$

Here, the description Hx≥0 means that all elements of the vector Hx arenon-negative. Further, this problem can also be deformed as representedby Expression (26) described below without losing generality.

[Math. 19]

minimize: x ^(T) Q _(within) x,

x ^(T) Q _(between) x=1, Hx≥0  (26)

This is known as a so-called nonconvex-quadratic constrained quadraticprogramming (QCQP) problem, is typically an NP difficult optimizationproblem, and takes a significantly long time to search for a solution.In order to achieve this optimization in a short period of time,alleviation to a semidefinite programming (SDP) is conducted.Specifically, xx<T> is replaced with a semidefinite value matrix X todeform the problem into an optimization problem that can be expressed byExpression (27) below.

[Math. 20]

minimize:

X,Q _(within)

_(F),

subject to:

X,Q _(between)

_(F)=1, HXH ^(T)≥0, X≥0, X ^(T) =X  (27)

Here, <A, B>F=tr(AB<T>) means a Frobenius inner product of the matrix,HXH<T>≥0 means that all elements of the matrix are non-negative, thatthe matrix X in Expression (28) described below included in Expression(27) is a semidefinite value, and X<T>=X is a symmetric matrix.

[Math. 21]

X≥0  (28)

For this problem, it is possible to obtain the optimized solution X at ahigh speed with an SDP solver. It is known from the number of SDPconstraining conditions that the rank of the solution is substantially1, and in that case, approximation of X=xx<T> can be performed.Therefore, the optimized solution vector u is obtained by obtaining aneigenvector corresponding to the maximum of the eigenvalue througheigenvalue decomposition of HXH<T>=Hxx<T>H<T>=uu<T>. Also, since u≥0 issecured when HXH<T>0 due to a theorem of Perron-Frobenius, it ispossible to use the optimized solution as it is on the assumption thatall the components are non-negative illumination setting values.

5.3: Third Modification Example

While there is a case in which it is difficult to apply a solving methodas a general eigenvalue problem according to the aforementioned secondmodification example, the third modification example is an example onthe assumption that the eigenvector can be a negative value. FIG. 7 is aschematic view illustrating an overview of a procedure for acquiring aninspection workpiece image according to the third modification example.As illustrated in the drawing, a case in which the illuminationparameter vector obtained as an optimized solution is x includingcomponents x1, x2, and x4 that are positive values and components x3 andx5 that are negative values, for example, as illustrated in FIG. 7 isassumed in the modification example. In this case, the optimizationcalculation unit 143 obtains a first optimized illumination lightemission pattern based on a vector x+=(x1, x2, 0, x4, 0) configured withcomponents that are positive values and obtains a second optimizedillumination light emission pattern based on a vector x-=(0, 0, −x3, 0,−x5) configured with absolute values of components that are negativevalues. Note that since x3 and x5 are negative values, the componentdescription of the vectors is −x3 and −x5. Then, the camera 102 and theimage-capturing unit 141 acquires a first image and second image byilluminating and imaging the workpiece to be inspected in the firstoptimized illumination light emission pattern and the second optimizedillumination light emission pattern, and the determination unit 144outputs an inspection workpiece image on the basis of a differencebetween the first image and the second image.

According to the configuration of the third modification example, it ispossible to obtain an optimized solution that mathematically completelyreproduces a maximum eigen vector of an eigenvalue in the generaleigenvalue problem and to reproduce an illumination intensity of anegative value in a pseudo manner. It is thus possible to obtain effectssimilar to those obtained by directly imaging a workpiece to beinspected in an optimized illumination light emission pattern based onan original optimized illumination parameter vector in which positivevalues and negative values are present together in a pseudo manner.

5.4: Fourth Modification Example

The fourth modification example is also an example on the assumptionthat an eigenvector can be a negative value and is an example in whichan imaging condition in multi-shot inspection is calculated using thecriteria of the Fisher's linear discrimination.

Generally, a projection vector is calculated as a maximum eigenvector ofan eigenvalue in a general eigenvalue problem in the Fisher's lineardiscrimination. However, it is possible to extract more information byregarding, as a feature portion space, a linear portion space configuredwith chosen M (L>M≥1) eigenvectors in order from a larger one as in maincomponent analysis as long as the eigenvalue is not zero. Note that itis possible to extract more information while a feature compression ratedecreases. However, if the value of M is set to be equal to or greaterthan the number L of illuminations, the setting is equivalent toacquisition of all pieces of information obtained from the multi-channelillumination, and therefore, there is no meaning in optimizing anillumination light emission pattern based on an idea of the Fisher'slinear discrimination. Also, since a cycle time increases as Mincreases, it is preferable to determine M on the basis of a cycle upperlimit that is allowable for the inspection process.

More specifically, in the fourth modification example, in a case inwhich illumination parameter vectors obtained as an optimized solutionincludes components of positive values and negative values, optimizationcalculation unit 143 chooses a plurality of (for example, M;corresponding to the number of dimensions of the feature portion space)eigenvectors of large eigenvalues in main component analysis as theillumination parameter vectors. Then, for each of the plurality ofeigenvectors, a first optimized illumination light emission patternbased on vectors configured with the components of positive values isobtained, and a second optimized illumination light emission patternbased on a vector configured with absolute values of the components ofnegative values is obtained.

Further, the camera 102 and the image-capturing unit 141 acquire a firstimage and a second image by illuminating and imaging the workpiece to beinspected in the first optimized illumination light emission pattern andthe second optimized illumination light emission pattern correspondingto each of the plurality of eigenvectors. Then, the determination unit144 acquires an inspection workpiece image on the basis of a differencebetween the first image and the second image thereof. Note thatprocessing of and after dividing vectors into vectors of the positivevalue components and vectors of absolute values of the negative valuecomponents in the processing in the modification example issubstantially equivalent to the processing in the third modificationexample illustrated in FIG. 7 other than that M eigenvectors are chosenand images are captured 2M times for positive and negative values.

Also, (A) of FIG. 8 and (B) of FIG. 8 are flowcharts illustrating anexample of processing procedures in an illumination optimization stageand an inspection stage according to the fourth modification example,respectively. The processing in the fourth modification example issimilar to the processing in the flowcharts illustrated in (A) of FIG. 6and (B) of FIG. 6 other than that the number M of dimensions of thefeature portion space is output in Step S412, an illumination lightemission pattern based on optimized illumination parameter vectors u1,u2, . . . , u2M for inspection is calculated in Step S414, andinspection workpiece images are captured in 2M optimized illuminationlight emission patterns at the time of inspection in Step S420.

According to the configuration of the fourth modification example, it ispossible to obtain effects equivalent to those obtained by directlyimaging a workpiece to be inspected in illumination light emissionpatterns (multiple shots) of a plurality of original illuminationparameter vectors in which positive values and negative values arepresent together in a pseudo manner. Also, since the configuration issubstantially for multi-shot inspection of a multiple number of images,it is possible to significantly enhance inspection performance ascompared with single-shot inspection. Further, since a dark currentoffset value is cancelled by using the difference between the firstimage and the second image, it is also possible to omit dark currentcorrection if black levels of both the images are the same.

5.5: Fifth Modification Example

The fifth modification example is an example in which a set of images asa result of multi-shot imaging are collectively handled as one hugecolumn vector since information obtained by a set of captured images inmulti-shot inspection is handled. In other words, in the fifthmodification example, the optimization calculation unit 143 performsillumination optimization using an illumination light emission patternfor evaluation corresponding to one column vector in which a pluralityof illumination parameter vectors configuring the illumination lightemission patterns for evaluation are superimposed in a case in which theillumination light emission pattern for evaluation is for multipleshots. Then, the optimization calculation unit 143 calculates anoptimized illumination light emission pattern for multiple shot thatmaximizes (an evaluation value of) the first index and minimizes (anevaluation value of) the second index.

When multi-shot inspection of M images is performed, one large vector gobtained by stacking m-th (1≤m≤M) captured image vectors gm are stackedin the column direction will be defined as Expression (29) describedbelow.

[Math. 22]

g=Σ _(m=1) ^(M) d _(m) ⊗g _(m)  (29)

In the above expression, each operator is as follows.

[Math. 23]

⊗: indicates a Kronecker product

d_(i): indicates a standard basis of

^(M)

The captured image gm of each of multiple shots is expressed asrepresented as Expression (30) described below by expressing imagesynthesis weights as wi and m.

[Math. 24]

g _(m)=Σ_(i=1) ^(N) w _(i,m) f _(i)  (30)

If these are collectively written again, it is possible to express theseas Expressions (32) and (33) described below using a multi-shot imagesynthesis weight matrix W of N×M that expresses an image synthesisweight vector in m-th imaging (corresponding to x in the aforementionedcase of single shot) by Expression (31) described below.

[Math. 25]

[W]_(i,m)∈

^(N)  (31)

g=Af  (32)

A=W ^(T) ⊗I, (HW≥0)  (33)

Var (S, m) in a case in which this definition is used is similarlyexpressed as Expression (34) described below and can be expressed as asecondary form of the illumination parameter vector similarly to thecase of the single shot. Note that vecW is expressed by Expression (35)described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 26} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {\underset{f\; \in \; S}{\mathbb{E}}{{g - {\underset{f\; \in \; S}{\mathbb{E}}g}}}^{2}}} \\{= {{{tr}\;{ARA}^{T}} = {{tr}\mspace{11mu}\left( {W^{T} \otimes I} \right){R\left( {W \otimes I} \right)}}}} \\{= {{{tr}\; W^{T}{QW}} = {\left( {{vec}\; W} \right)^{T}\left( {I \otimes Q} \right)\;\left( {{vec}\; W} \right)}}}\end{matrix} & (34) \\{{{vec}\mspace{11mu} W} = \left\lbrack {\lbrack W\rbrack_{i,1}^{T},\lbrack W\rbrack_{i,2}^{T},\ldots\mspace{14mu},\lbrack W\rbrack_{i,M}^{T}} \right\rbrack^{T}} & (35)\end{matrix}$

Here, if this situation is regarded as a situation in which theillumination parameter vector x in the case of single shot is replacedwith vecW, the dispersion matrix can be formulated as the same problemas that in the case of the single shot on the assumption that thedispersion matrix Q in the case of the single shot is replaced withExpression (36) described below in the case of the multiple shots.Non-negative conditions are handled with deformation as in Expressions(37) to (39) described below.

[Math. 27]

I⊗Q  (36)

HW≥0  (37)

(I⊗H)vec W≥0  (38)

(I⊗H)X(I⊗H ^(T))≥0  (39)

Then, each of the dispersion matrixes corresponding to the expectationvalues D_(between) and D_(within) is obtained as Expression (40)described below similarly to the case of the illumination optimizationin the single-shot inspection, and an optimal solution is obtained byreturning to the mathematical programming method using these dispersionmatrixes.

[Math. 28]

I⊗Q _(within) ,I⊗Q _(between)  (40)

According to the fifth modification example, one column vector is anillumination parameter matrix including a plurality of illuminationparameter vector, and it is possible to obtain an optimal illuminationlight emission pattern for multiple shots with arbitrary number ofcaptured images.

5.6: Sixth Modification Example

In optimization of illumination based on the criteria of the Fisher'slinear discrimination, it is possible to realize higher performance byoptimizing illumination parameters and image processing parameters atthe same time in a case in which linear image processing is applied toan inspection (label determination) stage of a workpiece to beinspected. Thus, the sixth modification example is an example in whichthe form of the image processing performed by the determination unit 144is linear image processing and optimization for maximizing the firstindex Db (D_(between)) and minimizing the second index Dw (Dwithin)performed by the optimization calculation unit 143 and optimization ofthe image processing parameter used in the image processing performed bythe determination unit 144 are performed at the same time.

Here, (A) of FIG. 9 and (B) of FIG. 9 are flowcharts illustrating anexample of processing procedures in an illumination optimization stageand an inspection stage according to the sixth modification example. Theprocessing in the sixth modification example is similar to theprocessing in the flowcharts illustrated in (A) of FIG. 6 and (B) ofFIG. 6 other than that Step S433 for allowing the user to designate animage processing type, for example, is executed in parallel with StepS412, the optimized illumination parameter vector u for inspection andoptimized image processing B for inspection are output in Step S414, andStep S442 of applying fixed linear image processing to the inspectionworkpiece image is executed prior to Step S422.

As examples of linear image processing, linear image processing shown inTable 2 is exemplified. Note that fin the table denotes a workpieceimage (input image) vector for inspection.

TABLE 2 Type of linear image processing Numerical expression Descriptionand effects Two-dimensional f[ 

,  

, c] * W W is a two-dimensional alignment convolution filter (* istwo-dimensional convolution). (Effect) Space frequency filterTwo-dimensional f[ 

,  

,  

] * w₁ · w₂ is one-dimensional alignment convolution filter (w₁ ⊗ w₂) 

(* is two-dimensional convolultion). (separation type) (Effect) Thenumber of parameters is reduced by limiting them to frequency propertiesindependent on the x axis and the y axis. Three-dimensional f * W W isthree-dimensional alignment convolutional filter (* is three-dimensionalconvolution). (Effect) Space frequency filter including colorsThree-dimensional f * (w₁ ⊗ w₂ ⊗ w₃) w₁ · w₂ · w₃ is one-dimensionalalignment convolutional filter (* is three-dimensional convolution).(separation type) (Effect) The number of parameters is reduced bylimiting them to frequency properties independent on the x axis, the yaxis, and colors. Color conversion w 

f[x, y,  

] w is a three-element vector. (Effect) Grayscale Color channel Wf[x, y, 

] W is an n × 3 matrix. number conversion (Effect) conversion of colorand the number of channels Partial space projection$\left( {\sum\limits_{\text{?} = 1}^{\text{?}}\;{\phi_{n}\mspace{11mu}\phi\text{?}}} \right)f$ϕ is an orthonormal basis. (Effect) It is possible to extract spatialmain components or remaining components using a PCA or ICA algorithm.Linear conversion Bf B is a matrix. (Effect) It is possible to configurelinear conversion including all the above elements.

indicates data missing or illegible when filed

As described above, although arbitrary linear image processing can bedescribed using a matrix expression B, constraining conditions betweenvariables within B differ depending on image processing types. Forexample, the image g after application of the image processing in thesingle-shot inspection can be expressed as Expression (41) describedbelow.

[Math. 29]

g=BΣ _(i=1) ^(N) x _(i) f _(i) =Af  (41)

Here, each of A and F represents a multi-channel image (one large columnvector) obtained by organizing projection matrixes defined byExpressions (42) and (43) described below and N images in the verticaldirection.

[Math. 30]

A=x ^(T) ⊗B  (42)

f=Σ _(i=1) ^(N) e _(i) ⊗f _(i)  (43)

In the above expressions, each operator is as follows.

[Math. 31]

⊗: indicates a Kronecker product

e_(i): indicates a standard basis of

^(N) ^(i)

Var(S, m) in the case in which the definition is used is similarlyexpressed as Expression (44) described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{11mu} 32} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {{\underset{f\; \in \; S}{\mathbb{E}}{{g - {\underset{f\; \in \; S}{\mathbb{E}}g}}}^{2}} = {\underset{f\; \in \; S}{\mathbb{E}}{{A\left( {f - {\underset{f\; \in \; S}{\mathbb{E}}f}} \right)}}^{2}}}} \\{= {{{tr}\;{ARA}^{T}} = {{{tr}\;\left( {x^{T} \otimes B} \right){R\left( {x \otimes B^{T}} \right)}} = {x^{T}Q_{B}x}}}}\end{matrix} & (44)\end{matrix}$

In the above expression, the matrix QB is specifically expressed asExpression (45) described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 33} \right\rbrack & \; \\\begin{matrix}{\left\lbrack Q_{B} \right\rbrack_{i,j} = {{tr}\mspace{11mu} B\;{\underset{f\; \in \; S}{\mathbb{E}}\left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)^{T}B^{T}}} \\{= {{\underset{f\; \in \; S}{\mathbb{E}}\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)}^{T}B^{T}{B\left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}}}\end{matrix} & (45)\end{matrix}$

Var(S, m) described here is not in the secondary form in regard to theillumination vector x and the image processing B, it is not possible toperform optimization at the same time by a method similar to that ineach example described hitherto. However, since a secondary form inregard to the illumination vector x is achieved by fixing the imageprocessing B, and on the other hand, a secondary form in regard to theimage processing B is achieved by fixing the illumination vector x, itis possible to achieve approximation to a result obtained in the case inwhich optimization is performed at the same time by iteratively andmutually repeating the optimization with each of the image processing Band the illumination vector x fixed. The optimization of theillumination vector x in the case in which the image processing B isfixed can be performed at a high speed by a mathematical programmingmethod such as SDP similarly to the aforementioned method throughreplacement of the definition of the matrix Q with QB.

Also, it is possible to handle optimization of the image processing B inthe case in which the illumination vector x is fixed by rewriting Var(S,m) as Expressions (46) and (47) described below.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 34} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {{x^{T}Q_{B}x} = {{\sum\limits_{i,j}{x_{i}{x_{j}\left\lbrack Q_{B} \right\rbrack}_{i,j}}} = {{tr}\;{BP}_{x}B^{T}}}}} \\{= {\left( {{vec}\mspace{11mu} B^{T}} \right)^{T}\left( {I \otimes P_{x}} \right)\left( {{vec}\; B^{T}} \right)}}\end{matrix} & (46) \\{P_{x} = {\sum\limits_{i,j}{x_{i}x_{j}{\underset{f\;\epsilon\; S}{\mathbb{E}}\left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)^{T}}}} & (47)\end{matrix}$

Generally, there are constraining conditions in the optimization of thematrix B of the image processing. In a case in which image processing isa convolution filter, for example, the shape of the matrix B isconstrained to a Toeplitz matrix, and in order to maintain brightness(DC component) at a constant value, a sum of coefficients is constrainedto be a constant value in many cases. Since constraining conditionschange depending on specific content of the image processing in theinspection stage, it is not possible to provide a specific generalizedexpression. However, with many of these, it is possible to obtain anoptimized solution at a high speed by similarly returning evaluationcriteria of the Fisher's linear discrimination to SDP. Also, theaforementioned algorithm is expressed as steps represented byExpressions (48) to (50) described below using a group of imageprocessing B that satisfies the constraining conditions.

[Math. 35]

Step (1)

B ⁽⁰⁾ =I  (48)

(Step 2)

Q′_(between) and Q′_(within) corresponding to QB are calculated on thebasis of B^((i)).

$\begin{matrix}\left( {{Step}\mspace{14mu} 3} \right) & \; \\{x^{(i)} = {\arg\;{\max\limits_{{{x}^{2} = 1},{{Hx} \geq 0}}\frac{x^{T}Q_{between}^{\prime}x}{x^{T}Q_{within}^{\prime}x}}}} & (49)\end{matrix}$

(Step 4)

P_(between) and P_(within) corresponding to Px are calculated on thebasis of x^((i)).

$\begin{matrix}\left( {{Step}\mspace{14mu} 5} \right) & \; \\{B^{({i + 1})} = {\arg\;{\max\limits_{B \in \mathcal{B}}\frac{{tr}\;{BP}_{between}B^{T}}{{tr}\;{BP}_{within}B^{T}}}}} & (50)\end{matrix}$

(Step 6)

i←i+1 is set and the processing is returned to Step 2 until theevaluation value is sufficiently converged.

In the case of multi-shot inspection, image processing is expressed as amatrix description B similarly to the case of single-shot inspection,and it is generally assumed that one column vector in which M imagevectors gm (1≤m≤M) are stacked is input as an input. The image g afterapplication of the image processing can be expressed as Expression (51)described below.

[Math. 36]

g=BΣ _(m=1) ^(M) d _(m) ⊗g _(m)  (51)

In the above expression, each operator is as follows.

[Math. 37]

⊗: indicates a Kronecker product

d_(i): indicates a standard basis of

^(M)

The captured image gm of each of multiple shots is expressed asExpression (52) described below by expressing an image synthesis weightas w_(i,m).

[Math. 38]

g _(m)=Σ_(i=1) ^(N) w _(i,m) f _(i)  (52)

If these are collectively written again, it is possible to express theseas Expressions (54) and (55) described below using a multi-shot imagesynthesis weight matrix W of N×M that expresses an image synthesisweight vector in m-th imaging (corresponding to x in the aforementionedcase of single shot) by Expression (53) described below.

[Math. 39]

[W]_(i,m)∈

^(N)  (53)

g=Af  (54)

A=W ^(T) ⊗B, (HW≥0)  (55)

In a case in which the definition is used, optimization of the N×Mmulti-shot image synthesis weight matrix W in a case in which the imageprocessing B is fixed is performed by developing Var (S, m) asExpression (56) described below. The definition of QB is equivalent tothat in the case of the single-shot inspection, this evaluationexpression is in the same form as that in the case in which no imageprocessing in the inspection stage is included, and optimization can beachieved by a similar method.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 40} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {{\underset{f\; \in \; S}{\mathbb{E}}{{g - {\underset{f\; \in \; S}{\mathbb{E}}g}}}^{2}} = {\underset{f\; \in \; S}{\mathbb{E}}{{A\left( {f - {\underset{f\; \in \; S}{\mathbb{E}}f}} \right)}}^{2}}}} \\{= {{{tr}\;{ARA}^{T}} = {{{tr}\mspace{11mu}\left( {W^{T} \otimes B} \right){R\left( {W \otimes B^{T}} \right)}} = {{tr}\mspace{14mu} W^{T}Q_{B}W}}}} \\{= {\left( {{vec}\; W} \right)^{T}\left( {I \otimes Q_{B}} \right)\;\left( {{vec}\; W} \right)}}\end{matrix} & (56)\end{matrix}$

It is possible to address the optimization of the image processing B inthe case in which the N×M multi-shot image synthesis weight matrix W isfixed by rewriting Var(S, m) as Expressions (57) and (58) describedbelow.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 41} \right\rbrack & \; \\\begin{matrix}{{{Var}\left( {S,m} \right)} = {{{tr}\mspace{11mu} W^{T}Q_{B}W} = {\sum\limits_{i,j}{\left\lbrack {WW}^{T} \right\rbrack_{i,j}\left\lbrack Q_{B} \right\rbrack}_{i,j}}}} \\{= {{{tr}\mspace{11mu}{BP}_{W}B^{T}} = {\left( {{vec}\; B^{T}} \right)^{T}\left( {I \otimes P_{W}} \right)\;\left( {{vec}\; B^{T}} \right)}}}\end{matrix} & (57) \\{P_{W} = {\sum_{i,j}{\left\lbrack {WW}^{T} \right\rbrack_{i,j}{\underset{f\; \in \; S}{\mathbb{E}}\left( {f_{i} - {\underset{f_{i}}{\mathbb{E}}f_{i}}} \right)}\;\left( {f_{j} - {\underset{f_{j}}{\mathbb{E}}f_{j}}} \right)^{T}}}} & (58)\end{matrix}$

Similarly, it is generally possible to obtain an optimized solution byreturning the optimization of the matrix B of the image processing toSDP or the like with constraining conditions employed in this case aswell. Note that since a method of alternately optimizing the multi-shotimage synthesis weight matrix W and the image processing B is alsoequivalent, detailed description will be omitted here.

Here, (A) of FIG. 16 to (C) of FIG. 16 are schematic plan viewsillustrating examples of the display screen of the user interfaceaccording to the sixth modification example (and the seventhmodification example, which will be described later). (A) of FIG. 16 isa pop-up window PW for selecting linear image processing, and a buttonBz for selecting a screen for setting coefficient parameters in variouskinds of linear image processing is also displayed. If a convolutionfilter is selected through tapping or the like from the pop-up windowPW, then the screen for selecting a type (Table 2) of linear imageprocessing as illustrated in (B) of FIG. 16 is displayed. If a desiredimage processing type is selected, a screen for selecting a predefinedcoefficient or a determination algorithm corresponding to simultaneousoptimization (sixth modification example) or the image processing typeis then displayed.

According to the sixth modification example, it is possible to handleboth single-shot inspection and multi-shot inspection and to optimize anillumination light emission pattern used in the inspection stage and thelinear image processing at the same time.

5.7: Seventh Modification Example

The seventh modification example is an example in which linear imageprocessing is performed on an evaluation workpiece image captured in theillumination optimization stage in illumination optimization based onthe criteria of the Fisher's linear discrimination. In other words, inthe seventh modification example, the optimization calculation unit 143performs linear image processing on the acquired evaluation workpieceimage prior to calculation of an optimized illumination light emissionpattern.

Here, (A) of FIG. 10 and (B) of FIG. 10 are flowcharts illustrating anexample of processing procedures in the illumination optimization stageand the inspection stage according to the seventh modification example.Processing in the seventh modification example is similar to theprocessing in the flowcharts illustrated in (A) of FIG. 9 and (B) ofFIG. 9 other than that a fixed parameter for linear image processingperformed in Step S254 performed later is designated in Step S433 andlinear image processing using the designated parameter is performed onthe evaluation workpiece image (or an image area) in Step S254 performedprior to Step S414.

Examples of the linear image processing performed here includeprocessing of causing images to conform by focusing only on a specificfrequency band or cutting DC components in order to ignore variations inoffset. The optimization method is equivalent to the method ofoptimizing only the illumination vector x with the image processing Bfixed similarly to the sixth modification example. Similarly to thesixth modification example, it is possible to perform desired linearimage processing, filter setting, and the like using the examples of thedisplay screen of the user interface illustrated in (A) of FIG. 16 to(C) of FIG. 16 in the seventh modification example as well.

According to the seventh modification example, it is possible to handleboth single-shot inspection and multi-shot inspection. Also, it ispossible to cause images to conform to each other or to be divergentfrom each other by focusing only on a specific frequency band, andfurther, it is possible to suitably cause images to conform to eachother or to be divergent from each other even in a case in whichprocessing of cutting DC components is included to ignore variations inoffset due to a change with time.

5.8: Eighth Modification Example

As described above, the illumination design based on the Fisher's lineardiscrimination is a method having evaluation criteria that differencesbetween non-defective products and defective-products are increased tocause defects to be easily recognizable and that variations in anon-defective product are reduced, and this is a method focusing ondifferences in a plurality of images. However, there may be a case inwhich it is more convenient to calculate an evaluation value from asingle image and to maximize or minimize the evaluation value thanhandling differences in images depending on problem setting (forexample, a case in which it is desired to simply increase or decreasecontrast in a specific area).

Thus, the eighth modification example is an example in which the imagearea setting unit 142 or the optimization calculation unit 143substitute an image in which all pixel values are zero in one of imagesin the region of interest (ROI) as comparison targets. In other words, aspecific optimization method in the case in which an evaluation value iscalculated from a single image in the eighth modification example can berealized by substituting an image in which all pixel values are zero toone of images to be compared in the case of the single-shot inspect inthe seventh modification example. Also, in a case in which contrast ishandled as an evaluation criterion, it is only necessary to set a highpass filter, for example, in the image processing B. More specifically,Var(S, m) is replaced with Expression (59) described below after thenumber of elements in the sample image are S is set to 1.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 42} \right\rbrack & \; \\\begin{matrix}{{{Var}(f)} = {{g}^{2} = {{{Af}}^{2} = {{tr}\mspace{11mu}{ARA}^{T}}}}} \\{= {{{tr}\;\left( {x^{T} \otimes B} \right){R\left( {x \otimes B^{T}} \right)}} = {x^{T}Q_{B}x}}}\end{matrix} & (59)\end{matrix}$

Also, since the final form of the expansion is the same, the procedurefor optimization in the case of single shot is the same as that in thecase of the seventh modification example. Also, the procedure for themulti-shot inspect is also the same as that in the case of themulti-shot inspection in the seventh modification example.

Also, as another variation, a pattern in which optimization to approacha prescribed image g₀ is performed is conceivable. In this case, Var(S,m) is replaced with Expression (60) below. Note that since a primaryform of x is placed in the evaluation expression in this case, it is notpossible to achieve optimization by exactly the same method as thatdescribed hitherto. Thus, specific examples of the optimization methodinclude utilization of a so-called nonconvex QP problem solver using anevaluation function of a difference type.

$\begin{matrix}\left\lbrack {{Math}.\mspace{14mu} 43} \right\rbrack & \; \\\begin{matrix}{{{Var}(f)} = {{{g - g_{0}}}^{2} = {{{Af} - g_{0}}}^{2}}} \\{= {{{tr}\;{ARA}^{T}} - {2\left( {{Af},g_{0}} \right)} + {g_{0}}^{2}}} \\{= {{{{tr}\left( {x^{T} \otimes B} \right)}{R\left( {x \otimes B^{T}} \right)}} - {2\left\langle {{\left( {x^{T} \otimes B} \right)f},g_{0}} \right\rangle} + {g_{0}}^{2}}} \\{= {{x^{T}Q_{B}x} + {2\left\langle {q_{B},x} \right\rangle} + {g_{0}}^{2}}}\end{matrix} & (60)\end{matrix}$

Here, variations of the evaluation expression from a single image thatcan be applied to the eighth modification example will be shown in Table3.

TABLE 3 Type of evaluation expression Numerical expression DescriptionSpecific frequency  

 ∥W * g∥² W is a two-dimensional or component size three-dimensionalalignment. * is a two-dimensional or three-dimensional convolution.Specific pattern size (size of a component belonging to a specificpartial space)${{\left( {\sum\limits_{\text{?} = 1}^{\text{?}}\;{\phi_{n}\mspace{11mu}\phi\text{?}}} \right)g}}^{2}$ϕ is an orthonormal basis. Coincidence of a patch ∥(B₁ − B₂)g∥² B1 andB2 correspond to operations of cut from image cutting regions from theimage g (the cutting sizes are assumend to be the same). Coincidencewith an ∥(g −  

)∥² g₀ is a prescribed template image. image template (* with a primaryterm) Coincidence of brightness |b 

g −  

|² b represents, as a vetor, an operation of (average color) (* with aprimary term) obtaining an inner product of a color vector obtained byan operation of averaging a prescribed range of the image g and aprescribed color vector. τ is a target QL value. Coincidence ofbrightness ∥Bg −  

∥² B is an operation of averaging a prescribed and color (* with aprimary term) range of the image g (a color vector is output). g₀ is atarget color vector. Coincidence of color ratio ∥bb 

Bg − Bg∥² B is an operation of averaging a prescribed range of the imageg (a color vector is output). b is a color ratio vector to be focused(standardized such that an L2 norm becomes one).

indicates data missing or illegible when filed

Here, (A) of FIG. 17 and (B) of FIG. 17 are schematic plan viewsillustrating examples of the display screen of the user interfaceaccording to the eighth modification example. In (A) of FIG. 17, animage in a region of interest (ROI) R0 designated by the user, an indexInd (including a weight) of the region of interest R0, and a touchbutton Bt for displaying a pop-up window PW for setting detailsillustrated in (B) of FIG. 17 are displayed. In the pop-up window PW in(B) of FIG. 17, various buttons for setting strength of a low passfilter LPF and a high pass filter HPF for the user to adjust a specificfrequency component in the region of interest R0 are displayed. Also, agauge GL indicating a set level of the low pass filter LPF and a gaugeGH indicating a level of the high pass filter HPF are disposed in thevicinity of the buttons. As described above, since the eighthmodification example substantially corresponds to evaluation using asingle image, the eighth modification example is useful in a case inwhich an evaluation value calculated from a single image is maximized orminimized.

5.9: Ninth Modification Example

It is possible to obtain a solution with averagely satisfactoryperformance for a plurality of sample image areas by the methodsdescribed hitherto. However, it is generally difficult to cause aplurality of conditions to be met at the same time since some of theconditions are competitive. Therefore, preparing a plurality of optimalsolutions with clear differences (inflections) through selection orexclusion of conditions to allow the user to perform comparison andconsideration, for example, is more advantageous in practice thanaveraging performance, depending on cases. Thus, the ninth modificationexample is an example in which weights are introduced into evaluationcriteria and user makes selection through comparison of a plurality ofsolutions obtained by causing the weights to change (deviate). In otherwords, in the ninth modification example, the optimization calculationunit 143 obtains a plurality of optimized illumination light emissionpatterns on the basis of a plurality of illumination parameter vectorsobtained as optimal solutions (suitable solutions) in a case in whichweights are applied to the first index Db and the second index Dw. Interms of optimization for multiple purposes, the optimal solutionsdescribed hitherto means Pareto optimal solutions.

More specifically, it is possible to add weights indicating importanceto sample images in an image group in calculation of a dispersion matrixusing the evaluation criteria of the Fisher's linear discrimination. Itis thus possible to keep a balance of importance among a plurality ofinter-image distances. Also, each weight may be adjusted using anevaluation expression to which a dispersion matrix with a meaning of “anevaluation expression from a single image” has been added in addition tothe dispersion matrix with the meaning of these inter-image distances.It is thus possible to keep a balance between importance of a pluralityof inter-image distances and importance of a plurality of evaluationvalues, each of which is from a single image.

Moreover, the weights may be able to be adjusted depending on theregions of interest (ROI). It is thus possible to collectively keep abalance of importance in accordance with differences in regions ofinterest. Also, the weights of these sample images may be adapted suchthat the dispersion matrix is a positive constant factor of I (unitmatrix) in a case in which all the weights are zero. It is thus possibleto keep a balance between maximization and optimization, that is, abalance between “coincidence of non-defective images” and “divergence ofdefective images”. Moreover, the aforementioned weight among sampleimages, the weight of the single image evaluation expression, the weightof the regions of interest, and the weight of the balance betweenmaximization and optimization are preferably designed to be mutuallyindependently adjustable. This facilitates the manual adjustment of theuser since each weight is regarded as an independent axis.

Then, a specific evaluation expression of a dispersion matrix thatsummarizes the above description and satisfies all the conditions willbe considered. Here, it is assumed that matrixes and vectors indicatingvarious optimization conditions are defined as shown in Table 4. Here,N_(p,k) is the number of samples of workpieces belonging to a designatedregion of interest and a label (class, category), and it is assumed that1≤s≤N_(p,k).

TABLE 4 Evaluation value Closeness of from a single images (relativeimage (absolute evaluation value) evaluation value) Criterion forQ_(within) ^((p, k, s)) Q_(within) ^(t(p, k, s)), q_(within)^((p, k, s)) minimization Criterion for Q_(between) ^((p, k))Q_(between) ^(t(p, k, s)), q_(between) ^((p, k, s)) maximization

“Closeness of images” is a square average (that is, dispersion) of adistance from an average image, and in a case in which N_(p,k)=2, thecloseness of images is equivalent to the distance between a pair ofimages. Also, weight coefficients shown in Table 5 are introduced tomerge these into one evaluation criterion. These can be set by the user.All the weight coefficients are assumed to be in a value range of equalto or greater than 0 and equal to or less than 1.

TABLE 5 Evaluation ROI Closeness of value from a comparison Entireimages single image target group intensity Criterion for α_(p, k, s)α′_(p, k, s) γ_(p) δ_(within) minimization Criterion for β_(p, k)β′_(p, k, s) γ_(p) δ_(between) maximization

At this time, examples of generating the matrixes that satisfy theaforementioned requirements will be represented as Expressions (61) to(64) described below. Note that it is also assumed that an averagevector m that is needed prior to the generation of each of the matrixesis similarly calculated with a weight.

$\begin{matrix}{\mspace{79mu}\left\lbrack {{Math}.\mspace{14mu} 44} \right\rbrack} & \; \\{Q_{between} = {{\prod\limits_{p,k}^{\;}{\left( {1 - {\delta_{between}\gamma_{p}\beta_{p,k}}} \right){\prod\limits_{s = 1}^{N_{p,k}}{\left( {1 - {\delta_{between}\gamma_{p}\beta_{p,k,s}^{\prime}}} \right)I}}}} + {\delta_{between}{\sum\limits_{p}{\gamma_{p}{\sum\limits_{k}\left( {{\beta_{p,k}Q_{between}^{({p,k})}} + {\sum\limits_{s = 1}^{N_{p,k}}{B_{p,k,s}^{\prime}Q_{between}^{\prime{({p,k,s})}}}}} \right)}}}}}} & (61) \\{\mspace{79mu}{q_{between} = {\delta_{between}{\sum\limits_{p}{\gamma_{p}{\sum\limits_{k}{\sum\limits_{s = 1}^{N_{p,k}}{B_{p,k,s}^{\prime}q_{between}^{({p,k,s})}}}}}}}}} & (62) \\{Q_{within} = {{\prod\limits_{p,k,s}{\left( {1 - {\delta_{within}\gamma_{p}\alpha_{p,k,s}}} \right)\left( {1 - {\delta_{within}\gamma_{p}\alpha_{p,k,s}^{\prime}}} \right)I}} + {\delta_{within}{\sum\limits_{p}{\gamma_{p}{\sum\limits_{k}\left\lbrack {{\sum\limits_{s = 1}^{N_{p,k}}{\alpha_{p,k,s}Q_{within}^{({p,k,s})}}} + {\alpha_{p,k,s}^{\prime}Q_{within}^{\prime{({p,k,s})}}}} \right\rbrack}}}}}} & (63) \\{\mspace{79mu}{q_{within} = {\delta_{within}{\sum\limits_{p}{\gamma_{p}{\sum\limits_{k}{\sum\limits_{s = 1}^{N_{p,k}}{\alpha_{p,k,s}^{\prime}q_{within}^{({p,k,s})}}}}}}}}} & (64)\end{matrix}$

The criterion for maximization is invalidated when δ_(between)=0 and isvalidated to the maximum extend when δ_(between)=1. Similarly, thecriterion for minimization is invalidated when δ_(within)=0 and isvalidated to the maximum extent when δ_(within)=1. The introduction ofthe weight coefficients described above enables the user to select,exclude, or balance a plurality of competitive evaluation criteria andto generate optimal solutions for a plurality of illuminationparameters.

Hereinafter, a method for presentation to the user for allowing the userto select a more optimal solution from the optimal solutions for theplurality of illumination parameters will be described. According to theevaluation criteria of the Fisher' linear discrimination, optimizationis performed with the Fisher's discriminant ratio (FDR) value that is aratio between the evaluation value of the criterion for maximization andthe evaluation value of the criterion for minimization. Althoughsuperiority and inferiority of the solutions are determined depending onhow large the criteria are, this information is not sufficient inpractice. In a situation in which there are solutions Ans1 and Ans2 andthe ratio value of Ans1 is better than the other, for example, asituation in which although the criterion for maximization of Ans1 ismuch better than that of Ans2, the criterion for minimization of Ans2 isbetter than that of Ans1 can be present. In a case in which a differenceof the ratio value between Ans1 and Ans2 is small, in particular, insuch a situation, it may be more effective to directly present thevalues of the criterion for maximization and the criterion forminimization to the user and allow the user to select one of them. Also,a difference between sample images that can be visually recognized inthe case of the solution Ans1 and in the case of the solution Ans2 maybe aligned and shown to the user to allow the user to select one ofthem.

Here, (A) of FIG. 18 is a schematic plan view illustrating an example ofthe display screen of the user interface according to the ninthmodification example. (A) of FIG. 18 illustrates an example of a pop-upwindow PW for displaying a result of setting the weight (aforementionedδ_(between)) of the first index Db and the weight (aforementionedδ_(within)) of the second index Dw, which are evaluation criteria, andperforming illumination optimization according to the ninth modificationexample. In the pop-up window PW, parameter sets Ps in which differentweights have been set for Db and Dw and gauges GB and GW indicating theweights B and W of Db and Dw in each parameter set PS are displayed.Also, a button BS for designating alignment (sorting) of the parametersets PS depending on the obtained ratio between Db and Dw or the ratioof the weights B and W thereof and a button BE for switching whether toapply the weights or not to apply the weights (all the weights are one)as an evaluation value of the Fisher's linear discrimination, forexample, are displayed below the parameter set PS in the drawing.

(B) of FIG. 18 is a flowchart illustrating an example of a processingprocedure according to the ninth modification example, and (C) of FIG.18 is a schematic plan view illustrating an example of a result selectedaccording to the ninth modification example. As illustrated in (B) ofFIG. 18, the weights of Db and Dw in each parameter set PS are adjustedin the pop-up window PW or an appropriate setting screen window first toexecute illumination optimization, and Db and Dw or the weights B and Wthereof in each parameter set PS are aligned and displayed in the pop-upwindow PW.

At this time, a light emission pattern of each channel illumination LSiof the light source LS, images corresponding to FIGS. 14 and 15, and thelike may be displayed together as reference images for the evaluationresult. Then, the user compares and considers the displayed result ofeach parameter set PS, and in a case in which there is a desiredsolution (Yes), the user selects the parameter set PS from which thesolution has been obtained and ends the optimization processing. In thiscase, the illumination light emission pattern of the light source LScorresponding to the selected parameter set SP (the parameter set 2 inthe drawing) and Db and DW or the weights B and W thereof in theparameter set PS may be displayed as illustrated in (C) of FIG. 18. Onthe other hand, in a case in which there is no desired solution (No),the user can adjust the weights again and repeat the processing ofexecuting the optimization.

According to the ninth modification example, it is possible to prepare aplurality of optimized solutions with clear differences (inflections)through selection or exclusion of conditions and to allow a user toperform comparison and consideration, for example. Also, it is possibleto cause how images are close to each other (degrees of coincidence) andan evaluation value from a single image to be present together asevaluation criteria or to set individual weights for how images areclose to each other (degrees of coincidence) and the evaluation valuefrom the single image. Moreover, it is also possible to individually setweights for differences in regions of interest. Also, a weight betweenimages for evaluation (sample images), a weight in a single imageevaluation scheme, weights of regions of interest, and a weight of abalance between maximization and optimization can be independentlyadjusted, and it is possible to allow the user to easily performadjustment.

5.10: Tenth Modification Example

The tenth modification example is a modification example of an imageshape of a region of interest (ROI). Generally, a case in which an imagein the region of interest is designated as a rectangular shape isassumed, and each portion in the region screen is handled withhomogeneous importance. However, the user may be able to set non-uniformweights in a region of interest using a paint brush or the like.According to such a tenth modification example, it is possible to adjustcoincidence, divergence, contrast, or the like of images while placingemphasis on a specific location in the region of interest.

Here, (A) of FIG. 19 and (B) of FIG. 19 are schematic plan viewsillustrating examples of the display screen of the user interfaceaccording to the tenth modification example. (A) of FIG. 19 displays animage in a region of interest (ROI) R0 designated by the user, an indexInd (including a weight) of the region of interest R0, and a touchbutton Bt for displaying the pop-up window PW for setting details asillustrated in (B) of FIG. 19. In the pop-up window PW in (B) of FIG.19, a paint brush group Bs used by the user to set non-uniform weightsin the region of interest R0 is disposed, and the image in the region ofinterest R0 is displayed again. The user can arbitrarily set non-uniformweights in the region of interest R0 by tapping and selecting a paintbrush with a desired size from the paint brush group Bs and tapping adesired portion in the region of interest R0 as it is.

§ 6 Appendixes

The embodiment and the modification examples described above have beenprovided for easy understanding of the present invention and are notintended to allow limited interpretation of the present invention.Elements included in the embodiment and the modification examples,disposition, materials, conditions, shapes, sizes, and the like thereofare not limited to those illustrated above and can be appropriatelychanged. It is also possible to partially replace or combine componentsdescribed in different embodiment and modification examples.

In other words, the aforementioned device for setting illuminationconditions when a target is inspected, the system, or a part thereof isrealized in the form of a software functional unit and is sold or usedas a single product, it is possible to store the software functionalunit in a computer-readable storage medium. In this manner, it ispossible to realize essences of the proposed techniques of the presentinvention, or portions that contribute to existing techniques, or anentirety or a part of the proposed techniques in the form of a softwareproduct and to store, in a storage medium, a computer software productincluding commands that causes a computer device (which can be apersonal computer, a server, a network device, or the like) to realizean entirety or a part of steps of the method described in each exampleof the present invention. The aforementioned storage medium can bevarious media capable of storing program codes, such as a USB, aread-only memory (ROM), a random access memory (RAM), a mobile harddisk, a floppy disk, or an optical disc.

Here, the present disclosure can also be expressed as follows inaccordance with the embodiments.

APPENDIX 1

An inspection device (1) including:

an image-capturing unit (100, 141) that has a light source (LS) thatilluminates at least one workpiece (4) including a workpiece (4) thathas areas corresponding to a plurality of different labels in aplurality of predetermined illumination light emission patterns and asensor (102) that images the at least one workpiece illuminated in theplurality of predetermined illumination light emission patterns andacquires a plurality of workpiece images for evaluation, each of whichis associated with each predetermined illumination light emissionpattern and each workpiece;

an image area setting unit (142) that extracts, for a combination of theplurality of different labels and at least one region of interest (R0 toR3) with a predetermined shape, images in the region of interest in theplurality of workpiece images for evaluation and sets image areas (or agroup);

an optimization calculation unit (143) which sets a first index (Db)indicating variations in workpieces (4) belonging to different labels onthe basis of varying components between images belonging to thedifferent labels, sets a second index (Dw) indicating variations inworkpieces belonging to the same label on the basis of varyingcomponents between images belonging to the same label, from the imagearea, obtains an illumination parameter vector as an optimal solutionthat maximizes the first index (Db) when the at least one workpiece (4)is illuminated and imaged in an illumination light emission pattern forevaluation corresponding to an arbitrary combination of the plurality ofpredetermined illumination light emission patterns and minimizes thesecond index (Dw), and calculates an optimized illumination lightemission pattern on the basis of the illumination parameter vector; and

a determination unit (144) which performs image processing on aninspection workpiece image obtained by the image-capturing unit (102,141) imaging the workpiece (4) to be inspected illuminated in theoptimized illumination light emission pattern to thereby determine alabel of the workpiece to be inspected.

APPENDIX 2

The inspection device according to Appendix 1, in which the optimizationcalculation unit obtains, through discriminant analysis, an optimizedsolution that maximizes an evaluation value of an evaluation function ofa ratio type based on a ratio between the first index (Db) and thesecond index (Dw) or an evaluation value of a difference type based onthe first index (Db) and the second index (Dw).

APPENDIX 3

The inspection device (1) according to Appendix 2, in which theoptimization calculation unit (143) includes a restriction conditionthat all components of the illumination parameter vectors arenon-negative values in the discriminant analysis.

APPENDIX 4

The inspection device (1) according to Appendix 2,

wherein in a case in which an illumination parameter vector obtained asthe optimized solution includes components of positive values andnegative values, the optimization calculation unit (143) obtains a firstoptimized illumination light emission pattern based on vectorsconfigured with the components of the positive values and obtains asecond optimized illumination light emission pattern based on vectorsconfigured with absolute values of the components of the negativevalues,

the image-capturing unit (102, 141) acquires a first image and a secondimage by imaging the workpiece (4) to be inspected with illumination inthe first optimized illumination light emission pattern and the secondoptimized illumination light emission pattern, respectively, and

the determination unit (144) acquires the inspection workpiece image onthe basis of a difference between the first image and the second image.

APPENDIX 5

The inspection device (1) according to Appendix 2,

wherein in a case in which an illumination parameter vector obtained asthe optimal solution includes components of positive values and negativevalues, the optimization calculation unit (143) chooses, as theillumination parameter vectors, a plurality of eigenvectors with largeeigenvalues in main component analysis, obtains a first optimizedillumination light emission pattern based on vectors configured with thecomponents of the positive values, and obtains a second optimizedillumination light emission pattern based on vectors configured withabsolute values of the components of the negative values for each of theplurality of eigenvectors for each of the plurality of eigenvectors,

the image-capturing unit (102, 141) acquires a first image and a secondimage by imaging the workpiece (4) to be inspected with illumination inthe first optimized illumination light emission pattern and the secondoptimized illumination light emission pattern corresponding to each ofthe plurality of eigenvectors, and

the determination unit (144) acquires the inspection workpiece image onthe basis of a difference between the first image and the second image.

APPENDIX 6

The inspection device (1) according to Appendix 1, in which when theillumination light emission pattern for evaluation is for multipleshots, the optimization calculation unit (143) calculates an optimizedillumination light emission pattern for multiple shots that maximizesthe first index (Db) and that minimizes the second index (Dw) when theat least one workpiece is illuminated and imaged in an illuminationlight emission pattern for evaluation corresponding to one column vectorobtained by superimposing the plurality of illumination parametervectors configuring the illumination light emission pattern forevaluation.

APPENDIX 7

The inspection device (1) according to any one of Appendixes 1 to 6,

in which a form of image processing performed by the determination unit(144) is linear image processing, and

optimization performed by the optimization calculation unit (143) tomaximize the first index (Db) and minimize the second index (Dw) andoptimization of image processing parameters used in image processingperformed by the determination unit (144) are performed at the sametime.

APPENDIX 8

The inspection device (1) to any one of Appendixes 1 to 7, in which theoptimization calculation unit (143) performs linear image processing onthe evaluation workpiece image prior to the calculation of the optimizedillumination light emission pattern.

APPENDIX 9

The inspection device (1) according to any one of Appendixes 1 to 8, inthe image area setting unit (142) or the optimization calculation unit(143), an image in which all pixel values are zero is substituted in oneof regions of interest to be compared.

APPENDIX 10

The inspection device according to any one of Appendixes 1 to 8, inwhich the optimization calculation unit (143) obtains a plurality of theoptimized illumination light emission patterns on the basis of aplurality of illumination parameter vectors obtained as the optimizedsolution when weights are applied to the first index (Db) and the secondindex (Dw).

APPENDIX 11

A method for inspecting a workpiece (4) using an inspection device (1)that includes an image-capturing unit (102, 141), an image area settingunit (142), an optimization calculation unit (143), and a determinationunit (144), the method including:

by the image-capturing unit (102, 141), illuminating at least oneworkpiece (4) including a workpiece that has areas corresponding todifferent labels in a plurality of predetermined illumination lightemission patterns and imaging the at least one workpiece (4) illuminatedin the plurality of predetermined illumination light emission patternsto acquire a plurality of evaluation workpiece images, each of which isassociated with each predetermined illumination light emission patternand each workpiece;

by the image area setting unit (142), extracting, for a combination ofthe plurality of different labels and at least one region of interestwith a predetermined shape, an image of the region of interest in theplurality of evaluation workpiece image and setting an image area;

by the optimization calculation unit, setting a first index (Db)indicating variations in workpieces (4) belonging to different labels onthe basis of varying components between images belonging to thedifferent labels and setting a second index (Dw) indicating variationsin workpieces (4) belonging to the same label on the basis of varyingcomponents between images belonging to the same label, from the imagearea, obtaining an illumination parameter vector as an optimal solutionthat maximizes the first index (Db) when the at least one workpiece (4)is illuminated and imaged in an illumination light emission pattern forevaluation corresponding to an arbitrary combination of the plurality ofpredetermined illumination light emission patterns and minimizes thesecond index (Dw), and calculating an optimized illumination lightemission pattern on the basis of the illumination parameter vector; and

by the determination unit (144), performing image processing on aninspection workpiece image acquired by the image-capturing unit (102,141) imaging a workpiece (4) to be inspected illuminated in theoptimized illumination light emission pattern to determine a label ofthe workpiece (4) to be inspected.

APPENDIX 12

A control program that causes a computer (100) to execute the imaging ofat least one workpiece, the setting of the image areas, the calculatingof the illumination light emission pattern for the optimization, and thedetermining of the label according to Appendix 11.

APPENDIX 13

A computer-readable non-transitory recording medium that records acontrol program for causing a computer (100) to execute the imaging ofat least one workpiece, the setting of the image areas, and thecalculating of the illumination light emission pattern for theoptimization, and the determining of the label according to Appendix 11.

REFERENCE SIGNS LIST

-   -   1 Inspection system    -   2 Belt conveyor    -   4 Workpiece    -   6 Imaging field of view    -   8 Upper network    -   10 PLC    -   12 Database device    -   100 Control device    -   102 Camera    -   104 Display    -   106 Keyboard    -   108 Mouse    -   110 Processor    -   112 Main memory    -   114 Camera interface    -   116 Input interface    -   118 Display interface    -   120 Communication interface    -   122 Internal bus    -   130 Storage    -   132 Programs for various kinds of processing    -   134 Illumination parameter    -   136 Learning machine parameter    -   138 Input image    -   140 Estimated image    -   141 Image-capturing unit    -   142 Image area setting unit    -   143 Optimization calculation unit    -   144 Determination unit    -   145 Storage unit    -   HL Scratch    -   LS Light source    -   LSi Channel illumination    -   PW Pop-up window    -   R0 to R4 Region of interest

1. An inspection device comprising: an image-capturing unit which has asensor that images at least one workpiece illuminated in a plurality ofpredetermined illumination light emission pattern and obtains aplurality of evaluation workpiece images that are associated with eachpredetermined illumination light emission patterns and each workpiece;an image area setting unit which sets, for the evaluation workpieceimages, a plurality of image areas associated with a plurality ofdifferent labels that indicate non-defective products or defectiveproducts; an optimization calculation unit which generates, from theimage areas, a first index, an output value of which increases asdifferences between image areas associated with labels indicating thenon-defective products and image areas associated with labels indicatingthe defective products increase, generates a second index, an outputvalue of which increases as differences between the image areasassociated with the labels indicating the non-defective productsincrease or as contrast in the image areas associated with the labelsindicating the non-defective products increases, and calculates anillumination light emission pattern for inspection such that the firstindex becomes larger and the second index becomes smaller; and adetermination unit which determines a pass/fail label of the workpieceto be inspected by performing image processing on an inspectionworkpiece image obtained by the image-capturing unit imaging a workpieceto be inspected illuminated in the illumination light emission patternfor inspection.
 2. The inspection device according to claim 1, whereinthe optimization calculation unit is adapted such that the number oflight emission patterns to be evaluated when the illumination lightemission pattern for inspection is calculated is larger than the numberof the predetermined illumination light emission patterns.
 3. Theinspection device according to claim 1, wherein the optimizationcalculation unit generates an evaluation function on the basis of thefirst index and the second index and obtains an optimized solution suchthat when a certain illumination light emission pattern for evaluationis provided, and when the first index becomes a maximum value and thesecond index value becomes a minimum value at the time of illuminatingand imaging the at least one workpiece, an evaluation value of theevaluation function becomes a maximum.
 4. The inspection deviceaccording to claim 1, wherein the optimization calculation unit obtains,through discriminant analysis, an optimized solution that maximizes anevaluation value of an evaluation function based on a ratio between thefirst index and the second index or an evaluation value of an evaluationfunction based on a difference between the first index and the secondindex.
 5. The inspection device according to claim 4, wherein theoptimization calculation unit includes a restriction condition that allcomponents of the illumination parameter vectors are non-negative valuesin the discriminant analysis.
 6. The inspection device according toclaim 1, wherein in a case in which an illumination parameter vectorobtained as an optimized solution includes components of positive valuesand negative values, the optimization calculation unit obtains a firstoptimized illumination light emission pattern based on vectorsconfigured with the components of the positive values and obtains asecond optimized illumination light emission pattern based on vectorsconfigured with absolute values of the components of the negativevalues, the image-capturing unit acquires a first image and a secondimage by imaging the workpiece to be inspected with illumination in thefirst optimized illumination light emission pattern and the secondoptimized illumination light emission pattern, respectively, and thedetermination unit acquires the inspection workpiece image on the basisof a difference between the first image and the second image.
 7. Theinspection device according to claim 1, wherein in a case in which anillumination parameter vector obtained as an optimal solution includescomponents of positive values and negative values, the optimizationcalculation unit chooses, as the illumination parameter vectors, aplurality of eigenvectors with large eigenvalues in main componentanalysis, obtains a first optimized illumination light emission patternbased on vectors configured with the components of the positive values,and obtains a second optimized illumination light emission pattern basedon vectors configured with absolute values of the components of thenegative values for each of the plurality of eigenvectors, theimage-capturing unit acquires a first image and a second image byimaging the workpiece to be inspected with illumination in the firstoptimized illumination light emission pattern and the second optimizedillumination light emission pattern corresponding to each of theplurality of eigenvectors, and the determination unit acquires theinspection workpiece image on the basis of a difference between thefirst image and the second image.
 8. The inspection device according toclaim 1, wherein in a case in which the illumination light emissionpattern for evaluation is for multiple shots, the optimizationcalculation unit calculates an optimized illumination light emissionpattern for multiple shots that maximizes the first index and minimizesthe second index when the at least one workpiece is illuminated andimaged in an illumination light emission pattern for evaluationcorresponding to one column vector obtained by superimposing theplurality of illumination parameter vectors configuring the illuminationlight emission pattern for evaluation.
 9. The inspection deviceaccording to claim 1, wherein a form of image processing performed bythe determination unit is linear image processing, and optimizationperformed by the optimization calculation unit to maximize the firstindex and minimize the second index and optimization of image processingparameters used in image processing performed by the determination unitare performed at the same time.
 10. The inspection device according toclaim 1, wherein the optimization calculation unit performs linear imageprocessing on the evaluation workpiece image prior to the calculation ofthe optimized illumination light emission pattern.
 11. The inspectiondevice according to claim 1, wherein the image area setting unit or theoptimization calculation unit makes both the first index and the secondindex respectively a total values of varying components of pixel valuesin images in regions of interest (ROI) with the same labels.
 12. Theinspection device according to claim 1, wherein the optimizationcalculation unit obtains a plurality of the optimized illumination lightemission patterns on the basis of a plurality of illumination parametervectors obtained as an optimized solution in a case in which weights areapplied to the first index and the second index.
 13. A inspection methodfor inspecting a workpiece using an inspection device that includes animage-capturing unit, an image area setting unit, an optimizationcalculation unit, and a determination unit, the inspection methodcomprising: an imaging step, by the image-capturing unit, imaging atleast one workpiece illuminated in a plurality of predeterminedillumination light emission patterns to acquire a plurality ofevaluation workpiece images, each of which is associated with eachpredetermined illumination light emission pattern and each workpiece; animage area setting step, by the image area setting unit, setting aplurality of image areas associated with a plurality of different labelsthat indicate non-defective products or defective products for theevaluation workpiece images; a calculation step, by the optimizationcalculation unit, generating a first index, an output value of whichincreases as differences between image areas associated with labelsindicating the non-defective products and image areas associated withlabels indicating the defective products increase, generating a secondindex, an output value of which increases as differences between theimage areas associated with the labels indicating the non-defectiveproducts increase or as contrast in the image areas associated with thelabels indicating the non-defective products increases, from the imageareas, and calculating an illumination light emission pattern forinspection such that the first index becomes larger and the second indexbecomes smaller; and a determination step, by the determination unit,determining a pass/fail label of the workpiece to be inspected byperforming image processing on an inspection workpiece image obtained bythe image-capturing unit imaging a workpiece to be inspected illuminatedin the illumination light emission pattern for inspection. 14.(canceled)
 15. A computer-readable non-transitory recording medium thatrecords a control program for causing a computer to execute the imagingof at least one workpiece, the setting of the image areas, and thecalculating of the illumination light emission pattern for theoptimization, and the determining of the pass/fail label according toclaim 13.